n+2 lookahead expert prefetching for Qwen Flash Next SSD streaming nets +8% on Mac
turtleninja99 · reddit · 2026-10-13
Optimizing Qwen Flash Next on a Mac Mini M5 64GB via SSD streaming, the author implemented an n+2 lookahead scheme suggested by the LocalLLama community: instead of fetching MoE expert weights one layer ahead, start loading them two layers ahead. Measured gains: +8% on chat and +4.2% on code. They note these stack on top of existing optimizations, so a vanilla setup could benefit even more. The project is open-sourced at github.com/skeggsguy/Flash-next-ssd.
More from Infra
- Real-time voice agent model survey: teams cut LLM latency from 1s to 0.3s by switching off GPT-4.1 — Natural-Chemical1513 · 2026-10-13
- Prediction: Opus 5.5 will run locally on a single 24GB GPU within 8 months — TheZachMueller · 2026-10-13
- Colibri: pure-C inference engine runs 2.8T-parameter MoE models by streaming experts from disk — tom_doerr · 2026-10-13
- Cloud vs On-Prem Is the Wrong Question for AI Agents: Split Workloads by Component — DavidLinthicum · 2026-10-13
- With RTX 5090 Discontinued, ComfyUI Users Hunt for Alternative GPUs — Impressive-Scene-562 · 2026-10-13
- Dev builds own peering network hitting 350Mbps in the wild, agent makes the landing page — vincent_koc · 2026-10-13