A $50,000 EPYC self-hosted K3 build targets 1.6 TB/s bandwidth and ~25 TPS
Liu_eroteme · x · 2026-07-29
The post describes a “sane” self-hosted K3 setup built around an EPYC 9556 and 16×256GB DDR5, with roughly $50,000 spent so far.
It argues that if you can source elusive 12800 MT/s MRDIMMs, you could get the full 1.6 TB/s memory bandwidth and, in vLLM terms, approach the 25 TPS theoretical ceiling for the setup. The reply frames the real bottleneck as making that memory stack available rather than just buying the CPU and RAM.
Related event: Kimi K3 2.8T-Parameter Model Runs on 80 RTX 5090s with Zero HBM(8 posts)→
More from Infra
- Unsloth Desktop Hotfix Adds Qwen-Image-2.1 Image Editing and Fixes GGUF Loading — danielhanchen · 2026-09-23
- Qwen 3.6 35B-A3B Q6 hits ~50 tok/s on a 128GB Strix Halo — what's the best local model now? — jankeydankey · 2026-09-23
- Together AI adds canary rollouts for zero-downtime model upgrades on dedicated inference — togethercompute · 2026-09-23
- Dedicated Hardware for Running AI Agents at Scale Arrives — cyrilzakka · 2026-09-23
- Ternary Bonsai 2 27B: 5.9GB weights retain ~95% of full-precision reasoning — cephaloform · 2026-09-23
- Qwen 27B runs 24hr unattended on one RTX5090, builds full Postgres-SpringBoot-React spreadsheet app — anglepoiselife · 2026-09-23