vLLM Upgrade Brings Major Server-Side Changes
vllm_project · x · 2026-07-12
Here are the serving, frontend, and upgrade notes for vLLM 0.25.0, focusing on engineering deployment:
- Distributed: sequence parallelism no longer requires DP, boosting end-to-end throughput by 1.9–5.0%; NCCL symmetric memory extended to AllGather / ReduceScatter; all2all adds fault tolerance to prevent output corruption.
- PD disaggregation: Introduces a secondary KV tier with Mooncake connector support; covers GDN (Qwen3.5) and MLA (DeepSeek-V4-Flash).
- Streaming Parser Engine: Unifies tool-call and reasoning parsing, adding Kimi k2.5 / k2.6 / k2.7 parsers.
- Rust frontend: Adds HTTPS/mTLS, DP supervisor, and profiler control routes.
- Quantization: Supports 2/3/5/6/7-bit weight-only inference (Humming) and Triton INT4 per-token-head KV cache quantization.
- Security fixes: Adds image decompression-bomb DoS guard, fixes infinite loops caused by NaN-audio, and limits tokenizer workload.
Important pre-upgrade notes: Legacy PagedAttention has been removed, V1/MRv2 is now the standard path, and certain models have been removed from the supported list.
More from Infra
- Nvidia Is Now Core to Every Major Robotaxi Stack at Commercial Scale — pdamodaran · 2026-09-11
- 12 KV Cache Reduction Techniques Every AI Engineer Should Understand, Explained — blaizedsouza · 2026-09-11
- The shadow GPU capacity market is formalizing, with Meta selling excess compute to outside buyers — DavidLinthicum · 2026-09-11
- Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs — bookwormengr · 2026-09-11
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11