Berkeley open-sources FreeToken: run 35B models on a $1,000 RTX 4060 laptop
Yuchenj_UW · x · 2026-08-22
UC Berkeley Sky Lab open-sourced FreeToken, dramatically speeding up local inference on consumer GPUs. A single RTX PRO 6000 workstation runs the 753B GLM-5.2 at 14.9 tok/s, an 8GB RTX 4060 laptop ($1,000) runs Qwen3.6-35B at 39.3 tok/s, and an RTX 5090 runs DeepSeek-V4-Flash 284B at 22-25 tok/s.
The key: it uses official checkpoints without extreme quantization, is 2-4x faster than Ollama, and lets you use frontier models with Claude Code or Codex locally for $0.
More from Infra
- MCP Isn't Replacing APIs: It's Changing Who APIs Are Designed For — kush_patil · 2026-08-22
- Data Center Opposition Surged from 42 to 75 Percent in One Year — The Decoder · 2026-08-22
- Qwen3.8-27B gets DFlash2 speculative-decoding GGUF release for llama.cpp — incoai · 2026-08-22
- Woolly post-trains Qwen3-8B for 2–3× faster math & code decoding — bosmeny · 2026-08-22
- Opinion: States Banning Data Centers Face 20 Years of Economic Depression — GabGarrett · 2026-08-22
- Vercel Fixes TLS Fragmentation Issue, All Websites Back Online — uwukko · 2026-08-22