vLLM serves DeepSeek-V4.1-Flash at 954K tokens/s per GPU on one GB300 NVL72 rack
bookwormengr · x · 2026-11-02
A poster suggests Elon Musk re-architect Grok around DeepSeek 4.1 Flash, citing 80x more token output per dollar of TCO. The linked vLLM blog backs the claim: with P/D disaggregation and Wide-EP on one NVIDIA GB300 NVL72 rack, vLLM hits 954K tokens/s per GPU at 196 tokens/s P90 per user on SemiAnalysis AgentX — 7.9x its earlier single-node result — with decode throughput within 6% of DeepSeek's own DeepInfer engine, at $38.96 per GPU-hour off-peak. Gains come from engineering on existing components (small DEP instances for prefill, Wide-EP with EPLB, a specialized low-latency decode path) plus DeepSeek's open kernels. Chart shows SemiAnalysis TCO curves for Kimi K3 vs DeepSeek V4.1 Flash.
More from Infra
- Texas Mega-Project Pipeline Tops $500B, Led by $100B Stargate AI Hub — MatthewChang · 2026-11-02
- SemiAnalysis: 30.3GW of planned 2028 US data center capacity at risk of pause after midterms — Eliv_nurotic · 2026-11-02
- B300 GPU rental prices plunge from $7.9/hr to $4.6/hr in a month as providers beg for customers — SumitGup · 2026-11-02
- FT chart: Big Tech's surging CapEx is a rational bet, not reckless spending — luisdans · 2026-11-02
- Google AI Ultra (€219.99/mo) user hits quota bug: locked out with 37% of 5-hour limit left — Specific-Cause-1014 · 2026-11-02
- Liquid AI's 2B d1 decision models beat same-size general models on classification benchmarks — JosephJacks_ · 2026-11-02