With 8-10T parameter models coming next year, who can still run frontier LLMs locally?
power97992 · reddit · 2026-10-01
A Reddit discussion on local runnability of next-gen frontier models: DeepSeek has said it will release an 8T model, Qwen plans a 10T one, and Kimi will likely follow, with flash models around 1-2T parameters.
The author estimates running a 4.4-bit 8T model with full context would take roughly nine 512GB M5 Ultras or 48 RTX 6000 Pros. Conclusion: only companies, cloud providers and the wealthy will run pro models locally; most people will struggle even with a 1T flash model and will likely settle for smaller models like Qwen 5 27B or the cloud.
More from Infra
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72 — bookwormengr · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16