Qwen 3.8 27B on M4 Max: 72.1 tok/s for code via ANE + MTP
TheMoonMidas · x · 2026-08-22
A developer shares an optimized recipe for running Qwen 3.8 27B on a Mac Studio M4 Max. By using the oMLX 0.6.3rc2 framework combined with ANE (Apple Neural Engine) for prefill acceleration and native Multi-Token Prediction (MTP, k=3), significant performance gains were achieved.
- Code generation: 72.1 tok/s
- Text generation: 53.3 tok/s
- Prefill speed: 274 tok/s (3.3× improvement)
This represents an 11% increase in decode speed over the previous best method. All tests were run locally without cloud APIs, and the raw benchmark data and configuration have been open-sourced on GitHub.
More from Infra
- SGLang's Weight Cache Daemon Cuts 1T Model Restart Time from 8.8 min to 32 sec — xiaosun86 · 2026-08-22
- Agent recursive loops blow up context costs: 5% failures eat 25% of bill — MaverikSh · 2026-08-22
- llama.cpp ships version 0.2.0 with official release notes — PhilippeEiffel · 2026-08-22
- Open Source Tool Mark Cleaner Locally Removes AI Text Watermarks and Metadata — VraserX · 2026-08-22
- Paper Reveals Larger LLMs Tolerate More Data Repetition During Pretraining — heghbalz · 2026-08-22
- Marin releases 23T-token pretraining dataset for public download — joecole · 2026-08-22