MLX MoE Layer Gets 1.5x Faster via Better Tile Scheduling in Grouped Matmul
awnihannun · x · 2026-09-28
A merged MLX PR reworks the gathermm kernel's Metal tile scheduling: previously tiles were assigned to thread groups independently of expert assignment, forcing extra matmul passes and redundant K scans (worst on short sequences). The new approach, inspired by quack's tile scheduler, aligns tile loading with expert assignment, delivering 1.5x speedup for MoE layers on Apple Silicon.
More from Infra
- Gemini 3.5 Flash now runs on air-gapped Google Distributed Cloud, offline up to 12 months — shashib · 2026-11-16
- Gemma.c: full Gemma 1/2/3 multimodal inference in a single C file, ~72 tok/s on CPU at INT8 — This-Purchase-3325 · 2026-11-16
- Sarvam AI Launches Token Factory: India-Hosted Inference for Open Models — itsOmSarraf_ · 2026-11-16
- BIS Issues Interim Final Rule Addressing Remote Access to Advanced AI Chips — apples_jimmy · 2026-11-16
- Jailbroken PS4 runs a 9B open-source model at up to 43 tok/s by talking directly to its 2013-era GPU — AIFlow_ML · 2026-11-16
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16