vLLM matches DeepInfer on DeepSeek-V4.1-Flash decode across batch sizes 8–256
woosuk_k · x · 2026-10-25
The vLLM team reports that on DeepSeek-V4.1-Flash, vLLM now matches DeepInfer — DeepSeek's own inference engine — across decode batch sizes 8 to 256, and does so at DEP32 versus DeepInfer's EP64DP64 configuration.
- DeepInfer was originally built on vLLM; by integrating DeepSeek's open-source kernels and heavily optimizing prefill and decode, vLLM has caught up on performance curves.
- vLLM core dev simonmo argues SemiAnalysis' AgentX is the benchmark to watch because it uses real traffic shapes, measured acceptance rates, and EPLB optimizations for DeepSeek models, translating directly to production performance.
- The attached SemiAnalysis chart estimates DeepSeek V4.1 Flash 552B economics on GB300 NVL72: at DeepSeek list pricing ($0.3/M input, $0.006 cached input, $1.2/M output), revenue of $50.43 per chip-hour with $45.43 profit — a 90.1% margin (3-year rent commitment, 60% utilization).
More from Infra
- Anthropic's noncancelable compute commitments hit $430B, per S-1 filing — Hesamation · 2026-10-27
- AI chip startup Tenstorrent raises $1.5B at $6.2B valuation, more than doubling — Beth_Kindig · 2026-10-27
- Anthropic discloses up to $120.5B SpaceX compute deal, ~$3.1B monthly at full capacity — XFreeze · 2026-10-27
- Satire: swapping Raft for an LLM-style decision model tanks write throughput 99.98% — irombie · 2026-10-27
- Cadence joins the domain-specific AI model camp for chip design — karlfreund · 2026-10-27
- Qdrant builds Universal I/O to run vector search mmap code on object storage — qdrant_engine · 2026-10-27