vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72
bookwormengr · x · 2026-11-16
vLLM and SGLang teams independently confirmed bookwormengr's math: at DeepSeek V4.1-Flash off-peak prices, Grace-Blackwell and Vera Rubin chips can yield $38 and $102 revenue per GPU-hour respectively. Per vLLM's blog, one GB300 NVL72 rack with P/D disaggregation and Wide-EP serves SemiAnalysis AgentX at 954K tokens/s per GPU (P90 196 tokens/s per user), a 7.9x jump over their single-node result, worth $38.96/GPU-hour. Decode throughput is within 6% of DeepSeek's own DeepInfer engine, proving a general-purpose engine can match a model-specific one. A Rubin peak operating point reaches 2.52M tokens/s/GPU ($102.91/GPU-h), 2.64x GB300 throughput.
More from Infra
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- Stratechery's Fictional Obituary: Imagining Jensen Huang's 2026 Exit From Nvidia — Gregory_C_Allen · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16