DeepSeek open-sources DeepInfer: one cluster serves 155T tokens/day at $38 per GPU-hour
Yuchenj_UW · x · 2026-10-21
DeepSeek officially open-sourced DeepInfer, its large-scale cloud inference framework, and disclosed rare operating numbers along with it: the framework powers a high-throughput, low-latency inference service for DeepSeek V4.1 Flash serving 155 trillion tokens/day on a single cluster, generating $38 per GPU-hour in revenue at off-peak pricing. This follows earlier open-sourcing of TileLang and inference operator libraries.
Yuchen Jin notes DeepInfer supports both NVIDIA Blackwell and Huawei Ascend (GPUs undisclosed), but even assuming rented B300s at $7/GPU-hour, the implied gross margin is 82% — effectively open-sourcing their revenue numbers too.
More from Infra
- RunInfra launches $10/mo coding plan: fastest open models across 19 agents incl. Claude Code — ycombinator · 2026-10-21
- Kimi K3 hits 391 tokens/s per user on TPU v7 with open-source vLLM megakernels — woosuk_k · 2026-10-21
- TPN Bench launches quantization service: I-matrix and quant-aware distillation for any HF model — const_reborn · 2026-10-21
- Turso unveils MVCC implementation to lift SQLite's single-writer bottleneck — glcst · 2026-10-21
- Railway pauses all new builds amid major outage, users ask if vibe coding is to blame — AaronBergman18 · 2026-10-21
- a16z: Superhuman AI hardware design is the easy part — manufacturing is the hard one — a16z · 2026-10-21