vLLM + NVIDIA wideEP Co-Design Makes GB300 NVL72 7.2x Better Perf per TCO
woosuk_k · x · 2026-10-15
SemiAnalysis reports that vLLM, Inferact and NVIDIA pushed co-design to the extreme on GB300 NVL72, achieving 7.2x better performance per TCO than other AI chips using a wideEP (wide expert-parallel) inference optimization that exploits the NVL72 backplane. The chart benchmarks DeepSeek V4.1 Flash 552B agentic tokens per $1 TCO vs P90 interactivity: GB300 NVL72 leads B200 (vLLM) by 7.22x and B300 by 3.21x, well ahead of MI325X/MI300X (vLLM). Hourly per-chip TCO listed: H100 $1.17, H200 $1.22, B200 $1.73, GB300 $2.31, MI300X $0.95.
More from Infra
- Anthropic ingests 1M+ telemetry and billing events per second via Google Pub/Sub — rseroter · 2026-10-15
- China May Not Even Hit 10 GW Total Accelerator Output by 2028, Argues Analyst — teortaxesTex · 2026-10-15
- China Now Builds ~0.5 GW of Accelerators Monthly, Bottleneck Shifting to Cluster MTBF — teortaxesTex · 2026-10-15
- Benchmarking coding agent harnesses and inference: lessons from $100Ks/month in research spend — abhijithneil · 2026-10-15
- Geometric self-attention: 14,000x smaller state than KV cache with better CE at 2K context — iskander · 2026-10-15
- Micron has returned 1,230% in three years, beating Nvidia, AMD and Broadcom 3-4x — Beth_Kindig · 2026-10-15