Ollama Auto-Enables Qwen3.5 MTP on Macs, MLX Backend Shows Major Speedup
BTA_Labs · reddit · 2026-08-05
Ollama's latest v0.32.6-rc0 release introduces a major inference optimization for Apple Silicon users: when a Qwen3.5 model includes built-in MTP (Multi-Token Prediction) tensors, Ollama’s MLX backend automatically loads the head and uses the model as its own speculative draft, requiring no extra configuration.
Adaptive Speculative Depth
The MTP head proposes one token per step, while Ollama’s controller dynamically adjusts the speculative depth based on measured cost and acceptance rate. The new build also logs detailed per-request stats, including tokens drafted/accepted, acceptance rate, and average/max draft depth.
Backend Discrepancy
Although Ollama simply notes that "Qwen3.5 is faster on Apple GPUs," the community noted that a previous llama.cpp Metal backend test showed an 11–27% regression using MTP (dropping from 25.3 tok/s to 19.3 tok/s on an M1 Max). The author points out that Ollama previously reported a 90% speedup for Gemma 4 using its MLX MTP implementation, suggesting the backend and adaptive depth controller matter more than MTP support itself. Standardized benchmarks are still needed to compare the real differences between MLX and Metal backends.
More from Infra
- a16z Podcast: Three Startups Reinventing US Tech Infrastructure — a16z Podcast · 2026-08-05
- Xiaohongshu Multimodal Inference Optimization: Vision Token Compression & MoE Re-routing — 小红书技术REDtech · 2026-08-05
- Performance Deep-Dive: Numpy and CPython in the Free-Threaded Build — abhi9u · 2026-08-05
- Discussing Groq's Ultra-Fast Inference: Does Real-World Speed Compromise Quality? — Scared-Tip7914 · 2026-08-05
- xAI Supercomputer in Memphis Coincides with 65% Surge in Housing Inventory — brianrkelly · 2026-08-05
- Dell's Son Raises $1B for BasePower, Valuing Home Battery Startup at $13B — 创业邦 · 2026-08-05