vLLM hits 829 P90 TPS single-user on four B300 GPUs, 26% faster than standard path on GB300
vllm_project · x · 2026-10-21
Following up on its DeepInfer comparison, the vLLM team shared latency numbers: on real AgentX sessions, vLLM's specialized decode path reaches 829 P90 TPS for a single user on four B300 GPUs, 26% faster than the standard path on GB300. The team also thanked DeepSeek for open-sourcing DeepInfer, with detailed numbers and reproduction recipes published.
More from Infra
- vLLM's specialized decode path hits 829 P90 TPS per user on four B300 GPUs, 26% faster — vllm_project · 2026-10-21
- vLLM says it can reproduce DeepSeek's inference performance, yielding $39/GPU/hr revenue — vllm_project · 2026-10-21
- vLLM matches DeepSeek's DeepInfer decode performance within 6%, generating $39/GPU/hr at off-peak pricing — vllm_project · 2026-10-21
- AMD brings EAGLE-4 speculative decoding to Instinct MI355X day 0, 1.6x faster than built-in MTP — hongyangzh · 2026-10-21
- TokenGate: learned token pruning speeds up LLM prefill up to 11.3x — Liana Patel · 2026-10-21
- Modal revives multiplayer game Curvytron to demo its new Sticky Sessions routing — charles_irl · 2026-10-21