vLLM hits 954K tokens/s per GPU serving DeepSeek-V4.1-Flash on GB300 NVL72, best AgentX result yet
woosuk_k · x · 2026-10-25
The vLLM team reports the best-ever DeepSeek-V4.1-Flash result on SemiAnalysis's AgentX real-world agentic workload.
- Architecture: P/D disaggregation plus Wide-EP scaled out on a GB300 NVL72 rack, 7.9× their single-node result
- Key numbers: 954K tokens/s per GPU total throughput (cached+input+output), 196 tokens/s P90 per-user interactivity
- Economics: $38.96/GPU-hour revenue at DeepSeek's off-peak pricing
- Also claims to beat DeepInfer's performance, with an engineering blog covering the optimization story and repro steps
SemiAnalysis's chart shows Dynamo vLLM on GB300 NVL72 far above GB200/SGLang, MI300X, H100 and other configs on the throughput-vs-interactivity curve.
More from Infra
- 8 mixed GPUs plus 256GB DDR4 vs DGX Spark: is this MoE inference rig viable? — Ambitious_Fold_2874 · 2026-10-25
- DeepMind paper: 5-second-per-layer quadratic simulations predict LLM training spikes — burny_tech · 2026-10-25
- Ontology-Driven Observability: Building an End-to-End Knowledge Graph at Netflix Scale — blaizedsouza · 2026-10-25
- Memory is 70-80% of AI rack silicon, and Rasgon asks what if prices never collapse — firstadopter · 2026-10-25
- GLM 5.3 Flash NVFP4 local-run repo assembled, tested up to 1M context (512K recommended) — HankYeomans · 2026-10-25
- Agents riding NVIDIA GPUs to unlock more NVIDIA GPUs: an Ouroboros moment — generativist · 2026-10-25