Qwen 3.8 27B on M4 Max: 72.1 tok/s for code via ANE + MTP
TheMoonMidas · x · 2026-08-22
A developer shares an optimized recipe for running Qwen 3.8 27B on a Mac Studio M4 Max. By using the oMLX 0.6.3rc2 framework combined with ANE (Apple Neural Engine) for prefill acceleration and native Multi-Token Prediction (MTP, k=3), significant performance gains were achieved.
- Code generation: 72.1 tok/s
- Text generation: 53.3 tok/s
- Prefill speed: 274 tok/s (3.3× improvement)
This represents an 11% increase in decode speed over the previous best method. All tests were run locally without cloud APIs, and the raw benchmark data and configuration have been open-sourced on GitHub.
More from Infra
- NVIDIA releases Nemotron 3.5 Bolt 74B MoE that fits in 24GB VRAM at 2-bit — NerdyRodent · 2026-10-16
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16