PyTorch 2.15 boosts Mac inference: MoE decoding up to 6.5x faster on Apple silicon
LysandreJik · x · 2026-10-20
The PyTorch team at Hugging Face has been replacing MPSGraph paths with native Metal kernels in the MPS backend, and PyTorch 2.15 delivers major speedups on Apple silicon:
- MoE decoding is 5-6.5x faster than 2.14
- Qwen3.6-35B-A3B: 9.4 → 61.3 tok/s on an M3 Ultra
- LFM2.5-8B-A1B: 23.6 → 123.6 tok/s
- Softmax over a 128k vocab is 8x faster
- FP8 is now supported on MPS
- Wan2.2 VAE decoding peak driver memory drops from 270 GiB to 8 GiB
Notably, this is not a separate runtime or inference engine — it's just PyTorch itself getting faster on Macs. Version 2.16 is expected to be even faster.
More from Infra
- Mac Studio with 256GB RAM now shipping in 15-17 weeks as demand outstrips supply — LinusEkenstam · 2026-10-20
- Google open-sources TorchTPU to run PyTorch workloads on TPUs with minimal code changes — rseroter · 2026-10-20
- Simular's neuro-symbolic agent runs 217x cheaper than Opus at 72% on OSWorld — xwang_lk · 2026-10-20
- AMD RX 6800 diffusion slowdown: smaller models run 8x slower than bigger ones under ROCm — GuaranteePurple4468 · 2026-10-20
- Ex-Google expert: data center capacity and electricity, not GPUs, are the real AI bottleneck — RihardJarc · 2026-10-20
- Shared Memory for 3 Parallel Research Agents Saved Nothing: A $1.11 Post-Mortem — quiet_flare · 2026-10-20