vLLM v0.30.0 ships with 762 commits: watermarking, HiSparse, Model Runner V2
vllm_project · x · 2026-09-23
vLLM v0.30.0 is out with 762 commits from 315 contributors (104 first-timers). Highlights:
- Hybrid-attention hot paths: Kimi K3 streamlines KDA/AtRes/MLA; DeepSeek-V4.1-Flash adds MXFP8 KV and async Engram; Qwen3.8-Flash-Next fuses QSA/PLE to cut sparse-GQA overhead
- HiSparse: adds a host tier beneath sparse-MLA decode; under GPU pressure only top-k misses return to a per-request hot buffer
- Model Runner V2: EAGLE3-style drafts for pipeline parallelism, adaptive verification extended to every draft-model speculator via online acceptance estimation
- Dual-key Gumbel-max watermark generation and detection, with per-request opt-out and speculative decoding support
- New models: GLM-5.3-Flash, K2-Horizon, Cohere Compass, Bailing V3 VL
- Fast Start: post-quantized TP-sharded weights resident in a per-GPU daemon, restarts map them via CUDA IPC with --load-format ipccache
Related event: vLLM v0.30.0 Released with 762 Commits and Major Performance Gains(3 posts)→
More from Infra
- Rivet launches SQLite Forking: 20ms agent-thread forks with zero data copying — samgoodwin89 · 2026-11-11
- French netizen proposes datacenters pay local residents compensation — clemnt · 2026-11-11
- GPU Rental Prices Hit Record Highs: B200 at $6.97/hr, H100 at $3.06 — inductionheads · 2026-11-11
- A Datacenter in Every Office: Building Work-Safe Local AI Racks — dee_hw · 2026-11-11
- "Compute is about power": the AI race's real bottleneck is energy, not chips — _sholtodouglas · 2026-11-11
- CoreWeave CEO says he canceled his 10b5-1 plan, halting insider selling — firstadopter · 2026-11-11