AMD lands MegaKernel decode support in upstream vLLM before NVIDIA
bookwormengr · x · 2026-10-13
Per SemiAnalysis, AMD has landed MegaKernel decode support in upstream vLLM well before NVIDIA — one of the first major vLLM features available only on ROCm with no CUDA equivalent. MegaKernels cut kernel launches 10x, a big win for high-interactivity decoding. CUDA vLLM doesn't even have draft PRs yet, though ASTRA is reportedly building a megakernel for Nvidia GPUs.
More from Infra
- Yoniq Compute repo ships inference recipes validated on 1x-8x RTX Pro 6000 with SGLang and vLLM — TheZachMueller · 2026-10-13
- RL post-training compute accounting: one weight update per episode blurs the train/inference line — davidmanheim · 2026-10-13
- Ex-Apple engineer eyeing Surface with NVIDIA RTX Spark: 128GB + CUDA in a backpack — LinusEkenstam · 2026-10-13
- Local 125B MoE on RTX 5090 finishes Three.js horse challenge in 23 minutes — Beneficial-Ad-8127 · 2026-10-13
- ML Veteran John Langford Endorses GPU MODE's Open Benchmark Competition Model — JohnCLangford · 2026-10-13
- Transcribing an hour of audio in ~13s on a MacBook Air — amrrs · 2026-10-13