vLLM v0.30 ships 762 commits: DeepSeek-V4.1-Flash support and 3.23x Gemma 4 multimodal speedup
vllm_project · x · 2026-09-23
vLLM released v0.30.0 with 762 commits from 315 contributors (104 first-timers).
New models
- DeepSeek-V4.1-Flash with all KV stored in MXFP8, a FlashMLA V4.1 record on SM100, DeepGEMM Mega-mHC, and async Engram prefetch
- DeepSeek-V4-Flash-Vision-Exp (ROCm, LoRA), GLM-5.3-Flash with EPLB, K2-Horizon, Cohere Compass, Bailing V3 VL, Nanbeige4.2
- A DeepSeek-V4 CPU backend with AVX512/AMX sparse MLA kernels
Performance
- RL sampling masks: GPU compaction restores throughput to within 2–3% of no-mask at 512 requests, fixing a 2x step-time regression
- Kimi K3 mixed batches: 5.2–7.7% higher E2E throughput
- Gemma 4 multimodal prefixes: 3.23x E2E speedup on RTX PRO 6000
- AMD ROCm packed W4A16 zero-points cut median TPOT 26.4% on Gemma 4 AWQ
Fast Start: a persistent per-GPU weight-cache daemon maps post-quantized, TP-sharded weights via CUDA IPC (--load-format ipccache), now covering FP4 and multi-node TP.
Upgrade notes: scale-out endpoints are now opt-in via --enable-scale-out, GPTQ gidx removed, Mamba cache deprecated, YaRN may reduce maxmodellen, default audio resampler is now torchaudio. Also adds Gumbel-max watermarked generation and detection.
Related event: vLLM v0.30.0 Released with 762 Commits and Major Performance Gains(3 posts)→
More from Infra
- tinygrad hits ~200 tok/s MiMo-V2.6-Pro on MI300X, brought up via GLM-5.3 — AIFlow_ML · 2026-09-23
- Rumors: 64GB+ VRAM RTX 5090 in R&D but not coming anytime soon — AIFlow_ML · 2026-09-23
- Swarm-built inference engine runs Qwen Image-2.1: 1K images in under 0.5s — bingxu_ · 2026-09-23
- MLX MTP head silently ignored: a 3-line fix boosts Mac local decode speed by up to 79% — Micha0827 · 2026-09-23
- Pluton: open-source self-hosted backup platform wrapping Restic and Rclone for encrypted cloud replication — tom_doerr · 2026-09-23
- TensorSharp's logit-reading approach beats LocalJev at structured decisions, 3.3x faster — fuzhongkai · 2026-09-23