vLLM says it can reproduce DeepSeek's inference performance, yielding $39/GPU/hr revenue
vllm_project · x · 2026-10-21
The vLLM project announced that with Wide-EP and EPLB, open-source vLLM now matches DeepSeek's inference performance: under DeepSeek's decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 stays within 6% of DeepInfer's published decode throughput across batch sizes 8–256.
On the AgentX agentic workload, vLLM pushes the Pareto frontier further on GB300, reaching up to 1M tokens/s/GPU and generating $38.96/GPU/hr in revenue at 196 P90 TPS under DeepSeek's off-peak API pricing. The team also extended low-latency kernel support for V4.1-Flash, hitting up to 826 TPS per user.
More from Infra
- DeepSeek's own numbers reveal ~82% gross margin: 155T tokens/day per cluster at $38/GPU-hour — ssh4net · 2026-10-21
- Running 32B-120B models locally on M1 Max 64GB: errors and model picks — Food4Lessy · 2026-10-21
- vLLM's specialized decode path hits 829 P90 TPS per user on four B300 GPUs, 26% faster — vllm_project · 2026-10-21
- vLLM matches DeepSeek's DeepInfer decode performance within 6%, generating $39/GPU/hr at off-peak pricing — vllm_project · 2026-10-21
- AMD brings EAGLE-4 speculative decoding to Instinct MI355X day 0, 1.6x faster than built-in MTP — hongyangzh · 2026-10-21
- TokenGate: learned token pruning speeds up LLM prefill up to 11.3x — Liana Patel · 2026-10-21