K3 Model Self-Hosting: $50K Hardware Setup Achieves ~25 tps Inference
Liu_eroteme · x · 2026-07-29
For self-hosting the K3 model (2.8T parameters), a hardware configuration is proposed:
- CPU: Epyc 9556
- RAM: 16x 256GB DDR5 (4TB total)
- Cost: $50,000
- With 12800MT/s MRDIMMs, theoretical ceiling 25 tps (1-200 tps with continuous batching)
Quoting @protosphinx, K3 weights are 1.4T (excluding KV cache). Moonshot recommends >64 accelerators for production. A 32x H200 cluster costs $1.2M, 64x $2.4M, with operational costs $15K/month.
Related event: Kimi K3 Runs on 80 RTX 5090s, Sparking Consumer-Grade Self-Hosting Debate(8 posts)→
More from Infra
- Extropic signs a $75 million Commerce Department LOI to scale thermodynamic AI chips — beffjezos · 2026-07-29
- SK hynix reportedly signs long-term contracts with about 10 major customers — dejavucoder · 2026-07-29
- Shanghai Aishengna is said to be manufacturing DUV lithography tools — zephyr_z9 · 2026-07-29
- A vendor-agnostic Vulkan backend cuts edge inference latency from 30 ms to 3 ms — ppchaos · 2026-07-29
- pdf-mcp turns technical PDFs into structured text, images, and searchable context — tom_doerr · 2026-07-29
- Moonshot’s Kimi K3 is a 2.8T open-weight MoE model with 1M-token context — alex_verem · 2026-07-29