FreeToken Lets 8GB GPUs Run 35B MoE Models On-Device
UC Berkeley and UT researchers released FreeToken, an inference engine that dynamically adapts GPU caches and bandwidth for MoE models, enabling an 8GB laptop GPU to run a 35B model at 39.3 tokens per second.
2026-08-29 ~ 2026-08-29 · 2 related posts
- Episode 1: UC Berkeley and MIT Open-Source FreeToken: Frontier MoE Models on Consumer PCs(2026-08-22, 11 posts)
- Episode 2: FreeToken Lets 8GB GPUs Run 35B MoE Models On-Device(2026-08-29, 2 posts)
- FreeToken Engine: Run 35B Models at 39.3 t/s on 8GB GPUs — rohanpaul_ai · 2026-08-29
- FreeToken: 8GB Laptop Runs 35B Model at 39.3 t/s via Edge-Native MoE Serving — rohanpaul_ai · 2026-08-29