vLLM v0.30 perf: 3.23x Gemma 4 multimodal speedup, ROCm TPOT -26.4%
vllm_project · x · 2026-09-23
Performance highlights from vLLM v0.30.0's release thread:
- RL sampling masks: GPU compaction restores throughput to within 2–3% of no-mask at 512 requests, fixing an 2x step-time regression
- Kimi K3 mixed batches: 5.2–7.7% higher E2E throughput
- Gemma 4 multimodal prefixes: 3.23x E2E speedup on RTX PRO 6000
- AMD ROCm: packed W4A16 zero-points cut median TPOT 26.4% on Gemma 4 AWQ
- Arm CPU: 6% higher throughput on Llama 3.1 8B W8A8
Related event: vLLM v0.30.0 Released with 762 Commits and Major Performance Gains(3 posts)→
More from Infra
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Eric Schmidt's foundation funds French NGO fighting against data centers — StephanSturges · 2026-11-12
- Epoch: Open-Weight GLM 5.3 Hacks Six Firefox Versions for ~$1,000 Each; Claude Fable 5.2 Sets ECI Record — Epoch AI · 2026-11-12
- Polymarket prices 28% odds of a state data center moratorium; Torus raises $500M — Polymarket · 2026-11-12
- AWS-wrapper cloud startup shuts down just one month after seed round — thedealdirector · 2026-11-12
- Utah startup Torus raises $500M for shipping container-sized AI data centers deployable in 4 months — Polymarket · 2026-11-12