MTPLX framework achieves ~2x speed boost for Qwen on Apple Silicon
koc_Z3 · reddit · 2026-08-29
The MTPLX framework claims to significantly accelerate Qwen models on Apple Silicon. Tests on an M1 Max 64GB show Qwen2.5-27B (Q4) decoding at 21 TPS and prefill at up to 111 TPS, a 2x boost. The framework features auto-tuning for draft depth and converts base models to MLX-ready MTP models.
More from Infra
- Magnitude open-source inference engine beats Ollama by over 2x with auto-tuned kernels — nickbaumann_ · 2026-10-15
- Nvidia to announce $28B in AI investments in Europe this week, report says — georgemillo · 2026-10-15
- A $10,811 Cloudflare Bill From an Unused AI-Written Test Project: Refunds Aren't a Safety Net — sujingshen · 2026-10-15
- Retraining-Free Method Turns Transformers Into Elastic Models Adapting to Any Compute Budget — VISLab-Amsterdam · 2026-10-15
- vLLM's MTP3 validation: AIME25 up to 0.9667, prefill kernels up to 84% lower latency — vllm_project · 2026-10-15
- Open-weights s1-llm-auto-router picks the right LLM in 26ms on a CPU — airesearch12 · 2026-10-15