Running 104GB Qwen3.8-Flash-Next on 48GB Mac at ~12 tok/s
yogthos · reddit · 2026-09-02
A Reddit user shared their experience running the Qwen3 8B Flash Next model (quantized to 104GB) on a 48GB Mac, achieving an inference speed of approximately 12 tokens per second.
The implementation relies on the slotstream project on GitHub. This demonstrates the viability of running larger parameter models on memory-constrained devices via quantization techniques.
More from Infra
- SGLang Summit sponsors grow from 11 to 36 in one week; Lip-Bu Tan, Lilian Weng to speak — BanghuaZ · 2026-10-18
- Yacine jokes his datacenter KPI is gigawatt-hours: get paged if not burning enough power — yacineMTB · 2026-10-18
- 1.68x inference speedup drops, directly accelerating GRPO post-training — yacineMTB · 2026-10-18
- OpenRouter is only a fraction of the total addressable market, poster argues — sloppenheimer · 2026-10-18
- Developer builds change alerts for LLM prices and retirements across Bedrock, Azure and Vertex — jsandhu0780 · 2026-10-18
- 12.3M tokens in 27 minutes for $0.14: agent session with 99.6% cache hit — teortaxesTex · 2026-10-18