Helion-powered vLLM linear backend on Hopper yields 1.11-1.18x kernel speedups
lmoroney · x · 2026-10-03
A new PyTorch blog post from Red Hat, Meta, and the vLLM teams shows Helion powering a linear backend for vLLM on NVIDIA Hopper:
- One Helion GEMM implementation covers Standard, Split-K, and Swap-AB variants, with an ahead-of-time autotuner picking the best config per shape
- Hybrid dispatch: Helion handles small token counts (up to 32), default CUTLASS or DeepGEMM handles larger ones
- Results: geometric-mean kernel speedups of 1.11x-1.18x over default libraries, and >10% end-to-end throughput gains on some workloads
- Code and autotuning tooling ship in their vLLM fork, with an RFC open for upstream discussion
The post frames it as a clean ML systems teaching case: express the algorithm once at a high level, let autotuning own shape-specific work, and keep the hot path under CUDA Graphs so tuning wins survive into serving.
Related event: PyTorch Helion Kernels Boost vLLM Inference Throughput(2 posts)→
More from Infra
- AMD Loom boosts Qwen3.5-4B prefill up to 2.6x on Radeon 890M laptops — jfowers_amd · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19
- Nvidia Allegedly Made GPUs Artificially Scarce to Keep Prices Sky-High — GaryMarcus · 2026-11-19
- AWS adds private CA support to Bedrock AgentCore Identity for connecting agents to self-hosted IdPs — AWS ML Blog · 2026-11-19
- Leak Hints at Classified 1M-GPU Training Cluster in China That Skips HBM — teortaxesTex · 2026-11-19