vLLM v0.33.0 ships with 927 commits, 1.5–2x faster decode on GB300
vllm_project · x · 2026-11-04
vLLM released v0.33.0 with 927 commits from 389 contributors (122 first-timers).
- New models: Reflection Beam-1.0, Step-5 Preview, Nemotron Parse 3.5, MiniCPM-OCR 1.0
- PyTorch 2.14: CUDA 13.0-only wheels, ABI-stable libtorch
- Spec decoding: XPress refiner, FLy lossy verify (1.04–1.31x), DSpark up from 1373 to 3390 tok/s on GB300
- NVIDIA decode: 1.5–2x lower latency on GB300, FA4 sparse MLA on Blackwell, 39–48% lower TTFT
- ROCm 10.0 now default for AMD, MonoKernel 2.32x P50
- New AWS EFA image and simplified vllm/vllm image name
More from Infra
- mlx-serve PR runs DeepSeek-V4.1 tensor-parallel across two Macs, 57% faster decode — HankYeomans · 2026-11-04
- PyTorch Day Japan 2026 agenda drops: ExecuTorch porting to IBM AIU edge accelerator — PyTorch · 2026-11-04
- LFM3 d1-2b decision model runs locally on Raspberry Pi at ~9s per decision — JosephJacks_ · 2026-11-04
- Inference Engine Explosion: llama.cpp Loyalist Can't Keep Up With New Frameworks — Kahvana · 2026-11-04
- One RTX 6000 Pro now out-earns the average human: $29,696/year in rent vs $23,000 global mean income — KyeGomezB · 2026-11-04
- Concurrency limits are the hidden tax in voice AI — how Phonic solved it — graceisford · 2026-11-04