TokenWing offloads 60% of MoE experts to SSD, 120B at 50 tok/s on a laptop

Grawnch · reddit · 2026-11-19

TokenWing, an open-source inference engine with a GUI wrapper called Run My AI, streams over 60% of routed MoE experts from SSD with low latency. User-reported benchmarks, all without MTP:

The project is early stage — the autotuner is still WIP, so manual flag tuning is recommended for now. Since the engine only reads (never writes), SSD wear is not a concern.

Original post →

More from Infra

Infra channel →