FreeToken: run 290B+ frontier MoE models locally on your gaming PC
Saboo_Shubham_ · x · 2026-08-25
FreeToken is an edge-native MoE serving engine (4.8k stars, 422 forks on GitHub) built to run frontier-scale open-weight models on consumer hardware — claiming interactive-speed inference of 290B+ parameter MoE models on a gaming PC.
Key design points:
- Treats heterogeneous edge resources (GPU, CPU, host memory, interconnects) as one elastic inference platform
- Bandwidth-adaptive CPU–GPU co-execution with a q policy
- Full-layer double-buffered prefill streaming, global LRU expert caching, and the FTW fast weight format
- Semantic-aware caching
It speaks the Anthropic/OpenAI APIs so Claude Code and Codex plug straight in; fully open source with a paper and community channels. The thread also highlights the Awesome LLM Apps repo (134k+ stars) with 100+ AI agent, skills and RAG templates.
More from coding & agent
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16
- Full course: build a playable Unity fishing game with Claude Opus 5.5 in six steps — Roger_M_Taylor · 2026-10-16
- Marc Lou praises PostShiba, a cute indie alternative to AWS SES for email infra — marclou · 2026-10-16
- RExBench: best coding agent completes only 33% of autonomous research extensions — najoungkim · 2026-10-16
- Wiring Pstack skills from a Grok bot into Hermes agent loops — alexcovo_eth · 2026-10-16
- past.dev launches long-term memory API for AI agents, tops BEAM public benchmark — thisiskp_ · 2026-10-16