vLLM serves DeepSeek-V4.1-Flash on Vera Rubin NVL72 at 58x GB300 throughput, 387 tokens/s per user
vLLM Blog · rss · 2026-11-12
vLLM's blog reports that on a single NVIDIA Vera Rubin NVL72 rack, vLLM serves DeepSeek-V4.1-Flash on the SemiAnalysis AgentX benchmark at 1.7M tokens/s per GPU while keeping a P90 of 387 tokens/s per user — 58x the throughput of GB300 NVL72 at the same interactivity.
The numbers showcase the new rack-scale Rubin architecture's throughput-latency balance on agent-style inference workloads, and confirm vLLM has been optimized for Vera Rubin.
More from Infra
- Japan inference startup ai& deploys heterogeneous chips, plans 100MW+ data centers — DavidBennett__ · 2026-11-13
- Accord GPU open beta queues ComfyUI, Blender, Ollama to stop VRAM fights — MrAddams_LibraLogic · 2026-11-13
- Ex-Nvidia CUDA engineer: 'CUDA is not the moat — Nvidia's only moat is its people' — AlpinDale · 2026-11-13
- AMD expands PsiQuantum partnership with planned equity investment in photonic quantum computing — sven_ai · 2026-11-13
- Nvidia downgrades hit seven as analysts say depths 'not fully probed' — SumitGup · 2026-11-13
- AMD Advancing AI World Tour Japan set for Dec 1 in Tokyo with KDDI, Microsoft, AMD execs — DavidBennett__ · 2026-11-13