vLLM matches DeepSeek's DeepInfer, verifying $38/GPU revenue claim; 7.9x throughput on GB300 NVL72
woosuk_k · x · 2026-10-25
vLLM reports that with P/D disaggregation and Wide-EP on a GB300 NVL72 rack, DeepSeek-V4.1-Flash inference throughput improves 7.9x, hitting 954K tokens/s per GPU (P90 196 tokens/s per user). Under SemiAnalysis's TCO model at DeepSeek off-peak prices, that works out to $38.96 revenue per GPU-hour — independently confirming DeepSeek's claimed $38/GPU economics. vLLM now matches DeepInfer (DeepSeek's in-house engine, itself built on vLLM) across batch sizes 8–256 at DEP32 vs DeepInfer's EP64DP64.
A widely shared take extrapolates the numbers: at this efficiency, 1GW of inference could generate $150B/year (conservatively $100B). If 25GW come online by end-2027, that implies up to $2.5T/year at DeepSeek rates or $10T at US-lab rates (8% of world GDP) — raising the question of whether the industry is overbuilding compute.
More from Infra
- xAI's Colossus 1 hits ~14.6 exaflops FP64, ~5x Top500 leader — cis_female · 2026-10-27
- Lambda runs 19 B200 nodes in a 16-node power budget, boosting throughput 24% — TheZachMueller · 2026-10-27
- Podcast: Micron's HBM roadmap, Meta's AI infra, and NVIDIA's supply chain lock-in — BenBajarin · 2026-10-27
- SGLang MegaKernel + PD Pushes the TCO-Latency Frontier on GB300, SemiAnalysis Benchmarks Show — AccBalanced · 2026-10-27
- Scaling self-hosted Convex: compiling TypeScript Convex functions to WASM — leventov · 2026-10-27
- Anthropic hits $16.3B H1 revenue as compute efficiency jumps 5.2x in 18 months — daniel_mac8 · 2026-10-27