Masked diffusion LLM on AMD XDNA2 NPU: 1.67x faster, 8.9x better energy than CPU
PyTorch · x · 2026-10-19
PyTorch officially previewed a PyTorchCon NA 2026 poster: AMD's Ravi Gupta, Shiksha Patel, and Rishi Madduri present efficient inference of masked diffusion language models on the AMD XDNA2 NPU. On a Ryzen AI MAX+ 395 APU at sequence length 128, the NPU path ran 504 ms per forward at 36 W vs 843 ms at 110 W for CPU INT4 — 1.67x faster and 8.9x better on energy. The poster also covers the INT4 quantization and graph-partitioning pipeline, the CastAvx fix, and the runtime approach for single-step prefill on the NPU. Rare real-world data on deploying diffusion LLMs on edge NPUs.
More from Infra
- Qwen3.8 27B benchmarks on NVIDIA RTX Spark: 19.3 tok/s decode, 100% accuracy — draginol · 2026-10-19
- Gemma 4 12b on NVIDIA RTX Spark laptop: 31.6 tok/s decode, 100% accuracy — draginol · 2026-10-19
- 4x RTX 5090 rig melts a 13A wall socket before the 12VHPWR plug — power-limit your cards — Zo0x78 · 2026-10-19
- NVIDIA shows how to scale model endpoints on Kubernetes with Dynamo in 5 minutes — NVIDIAAI · 2026-10-19
- Mac Studio M5 Ultra beats dual DGX Sparks in local LLM decode speeds — SumitGup · 2026-10-19
- Generating a 5:53 music video on an AMD 7900 XTX: any tips? — wgizmo · 2026-10-19