vLLM v0.30 ships 762 commits: DeepSeek-V4.1-Flash support and 3.23x Gemma 4 multimodal speedup
vllm_project · x · 2026-09-23
vLLM released v0.30.0 with 762 commits from 315 contributors (104 first-timers).
New models
- DeepSeek-V4.1-Flash with all KV stored in MXFP8, a FlashMLA V4.1 record on SM100, DeepGEMM Mega-mHC, and async Engram prefetch
- DeepSeek-V4-Flash-Vision-Exp (ROCm, LoRA), GLM-5.3-Flash with EPLB, K2-Horizon, Cohere Compass, Bailing V3 VL, Nanbeige4.2
- A DeepSeek-V4 CPU backend with AVX512/AMX sparse MLA kernels
Performance
- RL sampling masks: GPU compaction restores throughput to within 2–3% of no-mask at 512 requests, fixing a 2x step-time regression
- Kimi K3 mixed batches: 5.2–7.7% higher E2E throughput
- Gemma 4 multimodal prefixes: 3.23x E2E speedup on RTX PRO 6000
- AMD ROCm packed W4A16 zero-points cut median TPOT 26.4% on Gemma 4 AWQ
Fast Start: a persistent per-GPU weight-cache daemon maps post-quantized, TP-sharded weights via CUDA IPC (--load-format ipccache), now covering FP4 and multi-node TP.
Upgrade notes: scale-out endpoints are now opt-in via --enable-scale-out, GPTQ gidx removed, Mamba cache deprecated, YaRN may reduce maxmodellen, default audio resampler is now torchaudio. Also adds Gumbel-max watermarked generation and detection.
Related event: vLLM v0.30.0 Released with 762 Commits and Major Performance Gains(3 posts)→
More from Infra
- KubeCon keynote: computing is shifting from human speed to agent speed — HowDevelop · 2026-11-12
- Leaked memo: Meta infrastructure head Santosh Janardhan to leave in early 2027 — dinabass · 2026-11-12
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Eric Schmidt's foundation funds French NGO fighting against data centers — StephanSturges · 2026-11-12
- Epoch: Open-Weight GLM 5.3 Hacks Six Firefox Versions for ~$1,000 Each; Claude Fable 5.2 Sets ECI Record — Epoch AI · 2026-11-12
- Polymarket prices 28% odds of a state data center moratorium; Torus raises $500M — Polymarket · 2026-11-12