Grok does the math: running Kimi K3.1 Pro "locally" takes 18-32 H100s and $0.5-2M in hardware
giffmana · x · 2026-10-24
giffmana highlights a thread where Grok helped users reason from first principles about what "running a model locally" actually means, arguing trillion-parameter open models shouldn't be called "local."
- Kimi K3.1 Pro (2.8T total / 104B active MoE) needs 1.4-1.8TB VRAM even quantized: roughly 18-32 H100s
- Hardware costs $0.5-2M+; cloud rental starts $50/hr
- Practical interactive concurrency on such a setup: 20-40 users at 15-30 tok/s
- Only large enterprises can fund this; most small firms spend under $10k/year on AI
More from Infra
- AWS CEO Andy Jassy says 2026 capacity sold out, 2027 largely sold out, chunk of 2028 gone — sudoraohacker · 2026-10-24
- Every Mac on macOS 27 ships a free offline LLM in Terminal via the fm CLI — dr_cintas · 2026-10-24
- NVIDIA and UW Protein Design team up to release RFD4-Proteina for designed biology — AllThingsApx · 2026-10-24
- A ~€1600 local LLM rig: i7-14700KF + RX 9070XT hits 70-90 tokens/sec — Snoo-2768 · 2026-10-24
- Jensen Huang: Nvidia software licensing could add ~$20B/year at $2,000 per enterprise GPU — sudoraohacker · 2026-10-24
- The 'compute spread': H100 rental vs open-source LLM token prices show -0.46 correlation — AccBalanced · 2026-10-24