FreeToken Enables 35B Models on 8GB Laptops via Unified CPU-GPU Memory
mark_k · x · 2026-08-23
FreeToken is a software breakthrough for local AI that intelligently coordinates GPU, CPU, and system RAM to dynamically move MoE experts where needed, eliminating the requirement for enough VRAM to hold an entire model.
Benchmarks show an ordinary laptop with an 8GB RTX 4060 running a 35B model at 39 tok/s, while a gaming PC with an RTX 5090 can interactively run a massive 284B DeepSeek model. This allows users to run powerful AI on existing hardware without API costs or cloud dependency.
More from Infra
- Fluidstack hires Electrical Engineer for Modular R&D to build frontier AI infrastructure — MxMnr · 2026-10-16
- Jensen Huang joins BG2 podcast: does AI revenue justify capex, '27 bottlenecks — elonmusk · 2026-10-16
- Mistral's on-device infra rack revealed: DGX Spark, Mac Studio and more — gregmushen · 2026-10-16
- NVIDIA and Sweden's Wallenberg launch Sferical One, a shared AI supercomputer — nvidia · 2026-10-16
- Next infra frontier: self-evolving systems that adapt and self-verify — _shreya_s · 2026-10-16
- Xiaomi's MiMo RL dashboard reveals $3.47M spent on just 30 training steps — bycloud · 2026-10-16