Two tricks squeeze Qwen3.8-Flash-Next onto a 48GB MacBook Air
EyalToledano · x · 2026-08-28
Correction from the author: the 28 tok/s figure applies to his M4 Max, which has enough bandwidth; a MacBook Air will decode much slower, though the model still fits within the VRAM budget.
The core setup runs q4 on MLX using only 39GB of memory (q8 needs 50GB), stacking two tricks: REAP pruning cut the expert pool from 68GB to 35GB, and the 51B n-gram table now lives on SSD instead of RAM — his patch memmaps the shards and reads 100 bytes per lookup straight off disk before dequantizing.
Related event: Developer Prunes 180B-class Qwen MoE to Fit in 48GB of Mac Memory(9 posts)→
More from Infra
- SK Hynix ties new US plant to looser chip equipment exports to China, sources say — pstAsiatech · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15
- vLLM-Omni ships Day-0 serving for FLUX 3 Image Klein with BF16, FP8, NVFP4 support — vllm_project · 2026-10-15
- NVIDIA's Nemotron 3.5 Super VL lands training and inference support on Applied Compute — ypatil125 · 2026-10-15
- CoreWeave bond yield tops 16% as loans outlast customer contracts by ~2 years — tengyanAI · 2026-10-15