REAP Pruning + MLX Quantization Squeeze a 180B-Class Qwen Into 39GB
0xIlyy · x · 2026-08-28
HamsterResearch released Qwen3.8-Flash-Next-REAP-288-MLX-4bit: a 180B-class model that runs in just 39GB of memory, versus roughly 97GB VRAM for the stock version.
- MLX-native 4-bit quantization, 60% smaller than stock q4
- Experts pruned from 512 to 288 via REAP
- HumanEval holds at 91.5% (vs 93.9% stock)
This puts the model within reach of 48GB or 64GB MacBooks for local inference. Weights are on Hugging Face.
Related event: Developer Prunes 180B-class Qwen MoE to Fit in 48GB of Mac Memory(9 posts)→
More from Infra
- Jensen Huang: chips are just tools — sovereign AI means owning your data and knowledge — rohanpaul_ai · 2026-10-15
- Huawei Ascend to demo pretraining DeepSeek V4 on Ascend 950 supernodes in livestream — teortaxesTex · 2026-10-15
- Cloudflare data: after strikes on Yandex data centers, malicious bots simply moved networks — banteg · 2026-10-15
- OpenAI is hiring for its UK infrastructure team, offering end-to-end ownership — whoiskatrin · 2026-10-15
- XERJ indexes code, docs and PDFs in one command, saving agents 40x tokens vs grep — xerj-org · 2026-10-15
- TokenSpeed, a 'speed-of-light' LLM inference engine for Blackwell and Rubin, hits 2.3k stars — lightseekorg · 2026-10-15