Running open-weight models locally: Kimi K3.1 needs 594GB, ~$38k for 15 tok/s
ravikantagrawal · x · 2026-11-10
After months of benchmarking every new open-weight model (Kimi K3.1, DeepSeek V4.1 Flash, MiniMax M3.1), the author maps what local deployment actually takes: hardware needs, single-machine vs cluster performance, and when cloud compute wins — plus one-shot coding tests vs frontier models.
Kimi K3.1: a 512GB Mac Studio ($9.5k) doesn't fit (594GB minimum); four linked studios ($38k) yield 15 tok/s; AntSeed cloud 24 tok/s. Kimi K3.1 Pro weights alone are 1.56TB, needing four maxed M5 Ultra studios ($91k) just for memory.
More from Infra
- 10,000-hour vision-tactile dataset drops as robot data gold rush fuels shovel sellers — CyberRobooo · 2026-11-10
- Stratechery: Jensen Huang's Story Worsens — Governance Risk Looms Larger Than National Security — Stratechery · 2026-11-10
- InfraRSI Applies Recursive Self-Improvement to Training Infra: 2.75x Speedup, MFU Lifted from 17.4% to 30% — TokenRhythm · 2026-11-10
- ClickBench Playground Lets You Run SQL Against 110+ Databases Side by Side — mgill25 · 2026-11-10
- 4x RTX 3090 vs one RTX 5090: Malaysian builder weighs VRAM against newer architecture — whity2773 · 2026-11-10
- Hybrid-Spark switches drafters per decoding step, up to 1.62x faster than DSpark on code edits — Julianfmack · 2026-11-10