FreeToken Engine: Run 290B+ Frontier MoE Models Locally on 8GB Laptop GPUs
Saboo_Shubham_ · x · 2026-08-25
FreeToken is an edge-native Mixture-of-Experts (MoE) serving engine designed to run frontier-scale open-weight models on consumer hardware. It enables running frontier models on laptops with just 8GB VRAM, supports OpenAI and Anthropic API compatibility (e.g., Claude Code, Codex), and utilizes techniques like CPU-GPU co-execution and global LRU expert caching for optimization. The project is 100% open-source.
More from Infra
- SK Hynix ties new US plant to looser chip equipment exports to China, sources say — pstAsiatech · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15
- vLLM-Omni ships Day-0 serving for FLUX 3 Image Klein with BF16, FP8, NVFP4 support — vllm_project · 2026-10-15
- NVIDIA's Nemotron 3.5 Super VL lands training and inference support on Applied Compute — ypatil125 · 2026-10-15
- CoreWeave bond yield tops 16% as loans outlast customer contracts by ~2 years — tengyanAI · 2026-10-15