PyTorch Helion DSL integrated into vLLM beats CUTLASS and DeepGEMM with 10%+ throughput gains
PyTorch · x · 2026-10-03
PyTorch showcased Helion, its kernel DSL, by integrating it into vLLM's linear backend.
- A single Helion GEMM implementation covers multiple algorithmic variants — Standard GEMM, Split-K, and Swap-AB — with the best variant and config selected automatically per shape.
- On NVIDIA Hopper GPUs, per-shape tuning combined with hybrid dispatch outperforms vLLM's default CUTLASS and DeepGEMM backends across the evaluated models.
- The result is consistent end-to-end gains, with more than 10% throughput improvement on some workloads, in production inference settings.
- Written by Sean Chen (Red Hat) and Shangdi Yu (PyTorch, Meta).
Related event: PyTorch Helion Kernels Boost vLLM Inference Throughput(2 posts)→
More from Infra
- Investor: market hates uncertainty more than money after Nvidia earnings — MartinGTobias · 2026-11-19
- Microsoft and Intel deepen Azure collaboration: EMIB packaging, Diamond Rapids Xeons, 2.3x CXL gains — ryanshrout · 2026-11-19
- SGLang ships Day-0 serving support for new model DaisyRose-27B — ying11231 · 2026-11-19
- Linux LTS patch flood: 3318 patches for 6.6 alone, 'no way humans are reviewing this' — pkqzy888 · 2026-11-19
- DaisyRose-27B's in-house sandbox: ~80% cheaper than cloud providers, 99.95% success rate — BanghuaZ · 2026-11-19
- Tencent's 220B Hy4-flash quantized to 49 GB weights, runnable on 64 GB VRAM — gladkos · 2026-11-19