Berkeley-led Team Open-Sources FreeToken for Running MoE Models on Consumer Hardware

UC Berkeley, together with MIT and UT Austin, released and open-sourced FreeToken, a full-stack MoE serving system designed for consumer hardware, aiming to bring datacenter-scale open-weight models to personal devices. The project has drawn about 4.8k stars and 422 forks on GitHub since launch.

Confirmed

FreeToken's core techniques are bandwidth-adaptive execution, which dynamically splits computation between CPU and GPU based on real-time bus conditions, and lightweight semantic-aware caching that reuses computation across agent turns.
Official demo data: Qwen3.6-35B runs on an 8GB RTX 4060 laptop GPU, with preliminary benchmarks showing 2-4x speedups over Ollama.
The engine is compatible with OpenAI and Anthropic APIs, easing migration of existing applications.
Released as open source; the team claims it can run MoE models with 290B+ parameters at interactive speed on a gaming PC.

Unconfirmed

The "290B model on 8GB VRAM" claim comes from project promotion (relayed by @SabooShubham); public benchmarks only cover Qwen3.6-35B, so larger-scale performance awaits independent verification.

Why it matters

While MoE keeps active parameters small, the memory wall remains the main bottleneck for local deployment. By combining CPU-GPU co-execution with cache reuse, FreeToken offers a new path to run frontier open-weight models on consumer hardware.

2026-08-25 ~ 2026-08-26 · 5 related posts

Primary sources

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution — SteppenAxolotl · 2026-08-25 07:38 (UTC+08:00)
FreeToken Engine: Run 290B+ Frontier MoE Models Locally on 8GB Laptop GPUs — Saboo_Shubham_ · 2026-08-25 02:15 (UTC+08:00)
FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution — jiqizhixin · 2026-08-26 23:16 (UTC+08:00)

[source] FreeToken Engine: Run 290B+ Frontier MoE Models Locally on 8GB Laptop GPUs — Saboo_Shubham_ · 2026-08-25
FreeToken: run 290B+ frontier MoE models locally on your gaming PC — Saboo_Shubham_ · 2026-08-25
[source] FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution — SteppenAxolotl · 2026-08-25
UC Berkeley open-sources FreeToken: MoE inference 2-4x faster, runs 35B model on 8GB GPU — Roger_M_Taylor · 2026-08-25
[source] FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution — jiqizhixin · 2026-08-26