FreeToken Tested: 100 tok/s Running a 35B Model on Consumer Hardware
ViRROOO · reddit · 2026-08-22
A new project FreeToken (paper: arXiv:2608.16157, GitHub: FlashML-org/FreeToken) was released and the author ran initial local tests.
Setup: RTX 5080 (16GB VRAM), 64GB DDR6 RAM, AMD Ryzen 9 9950X3D—achieving 100 tokens/s on QWEN3.6-35B-A3B in NVFP4 quantization.
The author finds it impressive and invites the community to share their own benchmark results.
More from Infra
- NVIDIA and Sweden's Wallenberg launch Sferical One, a shared AI supercomputer — nvidia · 2026-10-16
- Next infra frontier: self-evolving systems that adapt and self-verify — _shreya_s · 2026-10-16
- Cerebras pushes disaggregated inference: heterogeneous prefill pools feeding wafer-scale decode — ai · 2026-10-16
- d-Matrix GA's Corsair rackscale, demos first public Attention-FFN disaggregation at 845 tok/s — ai · 2026-10-16
- Actual's inference engine hits 10k tok/s on NVIDIA DGX Station, GB300 case study — natesiggard · 2026-10-16
- TSMC Q3 revenue hits $46.7B, up 41% YoY; Q4 guidance of $52-54B tops estimates — Beth_Kindig · 2026-10-16