FreeToken Engine Runs 290B+ MoE Models on Consumer Hardware
wavefnx · x · 2026-08-19
FreeToken is an edge-native Mixture-of-Experts (MoE) serving engine designed to run frontier-scale open-weight models on consumer hardware. By leveraging bandwidth-adaptive CPU-GPU co-execution, double-buffered prefill streaming, and global LRU expert caching, it enables running 290B+ parameter models at interactive speeds on gaming PCs.
Related event: FreeToken: Edge-Native Engine Runs 290B MoE Models on Consumer Hardware(4 posts)→
More from Infra
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16
- Samsung cuts Q4 phone output as Nittobo shifts E-glass to AI-chip T-glass — zephyr_z9 · 2026-10-16
- White House AI chief discusses Empire AI, New York's shared compute infrastructure — mkratsios47 · 2026-10-16