vLLM hits 58x GB300 throughput on Rubin with DeepSeek-V4.1-Flash optimization
vllm_project · x · 2026-11-13
The vLLM team optimized DeepSeek-V4.1-Flash inference for Vera Rubin NVL72, reporting major gains on the SemiAnalysis AgentX benchmark:
- 1.7M tokens/s/GPU at P90 387 tokens/s/user — 58x GB300 NVL72
- Peak throughput of 2.52M tokens/s/GPU, 2.7x GB300
- 1,810 tokens/s with a single concurrent session using TP
Technically, the team reworked MoE, attention, GEMM, mHC and indexer kernels for Rubin: bandwidth-bound data stays in each SM's own memory domain, and SM107's wider MMA instructions do more work per issue, making prefill 28% and decode 25% faster. The specialized decode path was rebuilt for SM107 by fusing all-reduces into kernels, folding shared experts into routed GEMMs, and starting loads before waits — eliminating latency gaps in the kernel chain.
Related event: vLLM Optimizes Rubin: DeepSeek-V4.1-Flash Hits 58x GB300 Throughput(4 posts)→
More from Infra
- SGLang Summit agenda: LinkedIn serves ads model at 10K+ QPS under 80ms — DavidBennett__ · 2026-11-13
- Meme explains CUDA perfectly: optimized for many Alice Huangs, not one — djcows · 2026-11-13
- Perplexity's Mac app shows a local models option, hinting at offline capability — dnlkwk · 2026-11-13
- 2-bit DeepSeek V4.1 Flash runs a coding-agent seat on two 2016 V100s: a 12-hour real-work trial — a4mula · 2026-11-13
- IBM QDC 2026 Unveils Nighthawk r2 Quantum Chip and Fault-Tolerant Dev Tooling — MJBiercuk · 2026-11-13
- Why Jensen Huang Splits From US Frontier Labs on Open Source and Export Controls — morganlai · 2026-11-13