Challenge boosts Qwen 3.8 speed by 152% on Mac
gajesh · x · 2026-08-16
The Qwen MLX Challenge hosted by MLX.fast has achieved a breakthrough, demonstrating the potential of Apple Silicon for running dense models.
Optimization Results
- The current top score improved Qwen 3.8 27B inference speed on Mac by 152.5% over the baseline.
- Decoding speed reached 53.7 tokens/s, 2.5x faster than native multi-token prediction (MTP).
Technical Details
- Speculative decoding is key for dense models; solvers achieved gains by optimizing draft head weights.
- The scoring mechanism was updated to the median across eight hidden lengths to prevent overfitting.
- This result challenges the widespread belief that Macs are slow for dense models.
Related event: MLX Optimization Challenge Speeds Up Qwen 27B on Apple Silicon by Over 150%(4 posts)→
More from Infra
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16
- Cherry pi on a Raspberry Pi: the tiny-model rumor developers are watching — sull · 2026-10-16
- CoreWeave's debt costs falling: DDTL spreads tightened from S+9.62% to S+2.25% — BenBajarin · 2026-10-16
- SoftBank lines up US IPOs for SB Energy and AI robotics spinoff Roze, valuations top $50B — nordicinst · 2026-10-16
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16
- Hugging Face scales Indic pretraining with 2.8T tokens and an open data pipeline — Hugging Face Blog · 2026-10-16