Running Qwen Flash Next on a Mac Mini M5 64GB: SSD expert streaming benchmarked with charts
turtleninja99 · reddit · 2026-10-18
The author optimized Qwen Flash Next inference on a Mac Mini M5 64GB by streaming experts from SSD: a carousel buffering system keeps prompt processing nearly flat at lower expert cache, while a trained lookahead model prefetches experts to boost decode. Practical takeaways: 29 tks decode at max vs 26 after freeing 4GB; heavy prefill workloads can wind memory way down (12GiB cache max on 48GB Macs, 28GiB on 64GB); keep ≥18GiB free for macOS to avoid swap. Code and benchmarks open-sourced.
More from Infra
- krea2 LoRA training: 40-60 min on RTX 5090 vs 12 min on Fal.ai — Ambitious-Equal-7141 · 2026-10-18
- ComfyUI's new Assets feature scans external drives at startup without asking, user complains — Mad4reds · 2026-10-18
- First-Party Chips Are Mostly About Better NVIDIA Pricing, Fractile's Founder Says — No Priors · 2026-10-18
- ML engineer shows off his "messy but mine" homelab rack with 8 Noctua fans — TheZachMueller · 2026-10-18
- Ollama Cloud touts zero data retention, US/EU hosting, no training on your data — ollama · 2026-10-18
- One-click AI agent images on DigitalOcean now require 25–50GB minimum disks — vboykis · 2026-10-18