vLLM Upgrade Brings Major Server-Side Changes
vllm_project · x · 2026-07-12
Here are the serving, frontend, and upgrade notes for vLLM 0.25.0, focusing on engineering deployment:
- Distributed: sequence parallelism no longer requires DP, boosting end-to-end throughput by 1.9–5.0%; NCCL symmetric memory extended to AllGather / ReduceScatter; all2all adds fault tolerance to prevent output corruption.
- PD disaggregation: Introduces a secondary KV tier with Mooncake connector support; covers GDN (Qwen3.5) and MLA (DeepSeek-V4-Flash).
- Streaming Parser Engine: Unifies tool-call and reasoning parsing, adding Kimi k2.5 / k2.6 / k2.7 parsers.
- Rust frontend: Adds HTTPS/mTLS, DP supervisor, and profiler control routes.
- Quantization: Supports 2/3/5/6/7-bit weight-only inference (Humming) and Triton INT4 per-token-head KV cache quantization.
- Security fixes: Adds image decompression-bomb DoS guard, fixes infinite loops caused by NaN-audio, and limits tokenizer workload.
Important pre-upgrade notes: Legacy PagedAttention has been removed, V1/MRv2 is now the standard path, and certain models have been removed from the supported list.
More from Infra
- Nvidia previews Vera Rubin and takes aim at chiplet CPUs ahead of AMD's AI event — BenBajarin · 2026-07-22
- Nvidia doubles down on monolithic Vera CPU design for agentic workloads — BenBajarin · 2026-07-22
- Production AI budgets include retries, routing, caching and observability—not just token prices — arx-go · 2026-07-22
- NVIDIA briefs analysts on Vera CPU and doubles down on monolithic agentic design — BenBajarin · 2026-07-22
- NVIDIA unveils Vera Rubin platform with claims of 10x better performance per watt — nvidia · 2026-07-22
- SkyPilot comes out of stealth with a pitch to unify fragmented AI compute — skypilot_org · 2026-07-22