MLX bug fix nearly triples decode speed on M1/M2 Ultra Mac Studio

SumitGup · x · 2026-11-23

M1/M2 Ultra Mac Studio owners get a free local AI speedup: a small bug kept these chips on a slower decode path, ignoring an existing faster Metal kernel. A simple fix unlocked it — on an M1 Ultra 128GB running Qwen3-4B (4-bit), 8-row BF16 decode went from 28.05ms to 9.28ms, FP16 from 25.33ms to 8.55ms (3x), 4-row BF16 from 14.54ms to 9.32ms. Running 8 AI streams now costs nearly the same as 4, with no new kernel code needed.

Original post →

More from Infra

Infra channel →