MLX bug fix nearly triples decode speed on M1/M2 Ultra Mac Studio
SumitGup · x · 2026-11-23
M1/M2 Ultra Mac Studio owners get a free local AI speedup: a small bug kept these chips on a slower decode path, ignoring an existing faster Metal kernel. A simple fix unlocked it — on an M1 Ultra 128GB running Qwen3-4B (4-bit), 8-row BF16 decode went from 28.05ms to 9.28ms, FP16 from 25.33ms to 8.55ms (3x), 4-row BF16 from 14.54ms to 9.32ms. Running 8 AI streams now costs nearly the same as 4, with no new kernel code needed.
More from Infra
- Infinite Computing: How AI and Quantum Computing Converge Into One Fabric — ChuckDBrooks · 2026-11-23
- Inference practitioner on GPU co-ops: saturation math, weight caching, and quantization as dark art — sloppenheimer · 2026-11-23
- Nvidia's tale of two weeks: record revenue alongside regulatory probes and governance drama — emmanuelvivier · 2026-11-23
- Oracle raises $20B ($10B equity, $10B bonds) to fund OpenAI infrastructure deal — emmanuelvivier · 2026-11-23
- Forked PolyStrata hits ~1000 tok/s prefill for small model on RX 6700 XT — ireallydontcare00 · 2026-11-23
- DeepSeek on 4 DGX Sparks codes great with Opus 5.5 planning, GLM-5.3 not ready — AIFlow_ML · 2026-11-23