vLLM says it matches DeepSeek's inference performance, hitting 1M tokens/s/GPU on GB300

zhuohan123 · x · 2026-10-21

The vLLM project announced that with Wide-EP and EPLB, vLLM can reproduce DeepSeek's inference performance.

• Under DeepSeek's decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 achieves throughput within 6% of DeepInfer's published results across batch sizes 8–256
• On the AgentX workload, vLLM pushes the Pareto frontier further on GB300, reaching up to 1M tokens/s/GPU and generating $38.96/gpu/hr in revenue at 196 P90 TPS under DeepSeek's off-peak API pricing
• vLLM also extended low-latency kernel support for this setup

Related event: vLLM Says It Reproduces DeepSeek Inference Performance with Gains in Both Throughput and Latency(5 posts)→

Original post →

More from Infra

• Modal Engineers Detail Sticky Serverless Routing for Real-Time AI Inference — blelbach · 2026-10-21
• EdgeKernelBench: LLM-Generated Kernels Hit 64.5% Correct but Only 25.5% Fast on Edge Backends — encodelab · 2026-10-21
• Meta Open-Sources Dist-MoE, Speeding MoE Training Up to 1.40x on GB300 NVL72 — ezyang · 2026-10-21
• PyTorch accelerator integration group details progress and priorities at PyTorchCon — PyTorch · 2026-10-21
• Baseten ships four major open models day one, hits 668 tok/s on DeepSeek v4.1 Flash — baseten · 2026-10-21
• Report: 55% of China's AI compute goes to producing short dramas and videos — omoril0ver · 2026-10-21

Infra channel →