Patapsco AI open-sources cluster inference platform for very large models
tekbog · x · 2026-09-10
Patapsco AI released a beta open-source platform built over the past year for running multiple replicas of very large models across clusters. Features include KVCache-aware load balancing, API aggregation, a VSCode extension, a custom stream-caching filesystem, and a pailab tool to set up clusters-as-a-service. It scales from a single-node DGX Spark to a 100-node B200 cluster and is free to download, deployable directly on Kubernetes.
More from Infra
- Meta signs deal for tens of millions of AWS Graviton cores to power agentic AI — bookwormengr · 2026-09-10
- Edge0 open-sources streaming MoE framework with 35B and 8B on-device models — EAccelerate_42 · 2026-09-10
- iPhone 18 Pro's variable aperture: Apple returns to physics after hitting compute wall — r0ck3t23 · 2026-09-10
- Flux 2 Klein image model now runs fully in the browser via WebGPU — radamar · 2026-09-10
- AI server rail maker King Slide's founder is now Taiwan's richest man at $17.8B, passing Amodei and Altman — rwang07 · 2026-09-10
- SF Compute says resale of idle capacity saves customers 25%, cutting $4.5/hr to $3.375/hr — mattshumer_ · 2026-09-10