vLLM v0.29.0 cuts Blackwell E2E latency 33.6%, with 6.6-7.6x kernel speedups for Kimi-K3
vllm_project · x · 2026-09-11
vLLM's performance and hardware highlights for v0.29.0:
- Kimi-K3: Mamba metadata prep fused into a single Triton launch for 6.67.6x kernel-level speedup
- Batch invariance: tuned per architecture, 3x faster decode kernels on RTX 4090D / H20
- Blackwell autotuning: 33.6% E2E latency reduction
- NVIDIA: DeepSeek V3.2 and GLM-5.2 DSA reach the CUDA implementation on every GPU
- AMD ROCm: W4A4 now defaults to preshuffled asm GEMM; the ROCr/CLR base image fixes graph replay segfaults, up to 20% TPOT improvement at concurrency 1
- CPU: a new AMX MLA backend lets DeepSeek V2/V3/R1 run end to end
- Intel XPU: INC int4 W4A8 linear backend and AutoRound MXFP8 MoE support
Related event: vLLM v0.29.0 Cuts Blackwell Latency 33.6%, Model Runner V2 Default(4 posts)→
More from Infra
- Taiwanese families at TSMC's Arizona plant are shocked US high school is 'so much easier' — Polymarket · 2026-10-27
- OpenRouter hits 1 trillion tokens per day — ycombinator · 2026-10-27
- Engineer reverse-engineers Canon printer firmware to kill ink counters and proprietary cartridges — ngxson · 2026-10-27
- AI Cloud Operators Sell Years of Compute Yet Struggle to Finance Delivery — AccBalanced · 2026-10-27
- Modal engineer's deep dive into how vLLM and SGLang inference engines work — charles_irl · 2026-10-27
- TSMC reportedly preparing another CoWoS price hike of up to 20% as AI packaging demand outstrips supply — rohanpaul_ai · 2026-10-27