Berkeley open-sources FreeToken: run 35B models on a $1,000 RTX 4060 laptop
Yuchenj_UW · x · 2026-08-22
UC Berkeley Sky Lab open-sourced FreeToken, dramatically speeding up local inference on consumer GPUs. A single RTX PRO 6000 workstation runs the 753B GLM-5.2 at 14.9 tok/s, an 8GB RTX 4060 laptop ($1,000) runs Qwen3.6-35B at 39.3 tok/s, and an RTX 5090 runs DeepSeek-V4-Flash 284B at 22-25 tok/s.
The key: it uses official checkpoints without extreme quantization, is 2-4x faster than Ollama, and lets you use frontier models with Claude Code or Codex locally for $0.
More from Infra
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16
- Meta launches Muse Spark Open, says it runs on two M5 Ultra Mac Studios with Apple's help — The Verge AI · 2026-10-16