Frontier Models on Consumer GPUs: Qwen3.6 on 8GB VRAM at 39 tok/s
berkeley_ai · x · 2026-08-22
Optimizations for memory and compute now allow consumer GPUs to serve frontier-scale models at interactive speeds. Benchmarks show Qwen3.6 35B running at 39 tok/s on an 8GB RTX 4060 laptop, DeepSeek-V4-Flash 284B at 22-25 tok/s on an RTX 5090, and GLM-5.2 753B at 15 tok/s on an RTX PRO 6000. This significantly lowers the barrier to entry for deploying robust agentic workflows, enabling tools like Claude Code to run locally for $0.
More from Infra
- TSMC earnings note cross-referenced with WFE model and power checks in CS Atlas — BenBajarin · 2026-10-16
- Fluidstack hires Electrical Engineer for Modular R&D to build frontier AI infrastructure — MxMnr · 2026-10-16
- Jensen Huang joins BG2 podcast: does AI revenue justify capex, '27 bottlenecks — elonmusk · 2026-10-16
- Mistral's on-device infra rack revealed: DGX Spark, Mac Studio and more — gregmushen · 2026-10-16
- NVIDIA and Sweden's Wallenberg launch Sferical One, a shared AI supercomputer — nvidia · 2026-10-16
- Next infra frontier: self-evolving systems that adapt and self-verify — _shreya_s · 2026-10-16