DeepInfer Goes Open Source, vLLM Catches Up

Episode 1 · vLLM Matches DeepSeek's DeepInfer and Hits 954K tokens/s per GPU on GB300 NVL72 (2026-10-25, 10 posts)

vLLM officially announced that under DeepSeek-V4.1-Flash's official decode configuration, vLLM has matched the performance of DeepSeek's in-house inference engine DeepInfer across the entire batch size range from 8 to 256. The team also published an engineering blog post, "Serving DeepSeek-V4.1-Flash at Scale," and reported serving the model on a GB300 NVL72 rack under the SemiAnalysis AgentX real-world agentic workload at 954K tokens/s per GPU — claimed as the best-ever result for this model.

Confirmed

• Under DeepSeek's official decode configuration and across the full batch 8–256 range, vLLM's performance curve aligns with DeepInfer, achieved in DeepSeek's official evaluation environment (DEP3)
• On a single GB300 NVL72 rack, using P/D disaggregation (Prefill/Decode disaggregation) and Wide-EP, serving DeepSeek-V4.1-Flash for the SemiAnalysis AgentX benchmark reached 954K tokens/s/GPU
• Results were posted by vLLM's official account @vllmproject and amplified by co-founder @woosukk

Why it matters

• DeepInfer is DeepSeek's purpose-built inference engine for V4.1-Flash; the open-source vLLM framework matching its performance under official configurations means users can get equivalent throughput without a proprietary engine, significantly lowering the barrier to large-scale deployment
• The 954K tokens/s per GPU figure on a single GB300 NVL72 rack demonstrates the scalability of the new NVL72 hardware combined with P/D disaggregation on real agentic workloads, offering a reference benchmark for hyperscale inference serving

See field notes for sourcerefs placeholder.

• vLLM matches DeepInfer on DeepSeek-V4.1-Flash across batch sizes 8–256 — vllm_project · 2026-10-25
• vLLM hits 954K tokens/s/GPU on GB300 NVL72, matching DeepSeek's DeepInfer — vllm_project · 2026-10-25
• vLLM hits 954K tokens/s per GPU serving DeepSeek V4.1-Flash on GB300 NVL72 rack — vllm_project · 2026-10-25
• vLLM matches DeepInfer on DeepSeek-V4.1-Flash decode across batch sizes 8–256 — woosuk_k · 2026-10-25
• vLLM hits 954K tokens/s per GPU serving DeepSeek-V4.1-Flash on GB300 NVL72, best AgentX result yet — woosuk_k · 2026-10-25
• vLLM now matches DeepSeek's in-house DeepInfer engine on DeepSeek-V4.1-Flash decode — lmoroney · 2026-10-25
• vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU on GB300 NVL72 rack — woosuk_k · 2026-10-25
• vLLM matches DeepSeek's DeepInfer, verifying $38/GPU revenue claim; 7.9x throughput on GB300 NVL72 — woosuk_k · 2026-10-25
• vLLM serves DeepSeek-V4.1-Flash at 954K tokens/s per GPU, matching DeepInfer at 7.9× throughput — Xianbao_QIAN · 2026-10-25
• vLLM matches DeepSeek's DeepInfer engine on V4.1-Flash, verified by SemiAnalysis — AccBalanced · 2026-10-26

Episode 2 · DeepSeek Open-Sources DeepInfer, Its Production Inference Engine for V4.1 Flash (2026-10-25, 6 posts)

DeepSeek has open-sourced DeepInfer, the inference engine that actually powers its V4.1 Flash service in production, making it one of the most closely watched AI infrastructure stories of the cycle. Released under Apache 2.0, the project supports both NVIDIA Blackwell and Huawei Ascend hardware.

Confirmed

• DeepInfer is the inference framework DeepSeek actually runs in production, serving V4.1 Flash online, and is open-sourced under Apache 2.0 (@altryne).
• On the hardware side, it is compatible with both NVIDIA Blackwell and Huawei Ascend chips (@altryne).
• Performance and economics figures relayed by @dlweekly: a single cluster serves 155 trillion tokens per day; at off-peak pricing, revenue works out to roughly $38 per GPU hour.
• The ThursdAI podcast (@thursdaipod) did an item-by-item breakdown of DeepInfer, analyzing what it includes and what's still missing; guest Yam Peleg said the release "broke the internet."

Why it matters

• DeepInfer tackles the core problem of inference frameworks—serving massive user bases at reasonable cost (@dlweekdly)—and by open-sourcing a production-grade system rather than demo code, it offers direct reference value to the inference infrastructure community.
• Supporting both NVIDIA and Huawei Ascend hardware means the engine provides a reusable production-grade inference path for the domestic Ascend ecosystem.
• The roughly $38-per-GPU-hour revenue figure offers a rare public anchor for outsiders estimating the economics of large-model inference services.

• DeepSeek open-sources its production inference engine for V4.1 Flash, runs on Blackwell and Ascend — altryne · 2026-10-25
• DeepSeek open-sources DeepInfer, the engine serving V4.1 Flash, under Apache 2.0 — altryne · 2026-10-25
• DeepSeek open-sources DeepInfer, the inference engine behind V4.1 Flash — thursdai_pod · 2026-10-25
• ThursdAI breaks down DeepSeek's open-sourced DeepInfer inference engine — thursdai_pod · 2026-10-25
• DeepInfer is deeply tailored to DeepSeek 4.1F: KV cache of just 890 bytes per token — teortaxesTex · 2026-10-25
• DeepSeek open-sources DeepInfer: cluster serves 155T tokens/day at $38 per GPU-hour — dl_weekly · 2026-10-26