DeepSeek open-sources DeepInfer, claims ~80% serving margin at $38/card-hour
dotey · x · 2026-10-21
DeepSeek open-sourced DeepInfer, its in-house inference framework powering DeepSeek-V4.1-Flash at scale. Per its Zhihu writeup (flagged by swyx), a single cluster serves 155T tokens/day at $38 per card-hour revenue against an estimated $5-10/GPU-hour cost — implying an 80% serving margin. The stack targets multi-vendor chips, 8-card Prefill / 64-128-card Decode supernodes, and 72h+ KVCache retention to cut repeated prefill compute.
Related event: DeepSeek Open-Sources DeepInfer Inference Framework(40 posts)→
More from Infra
- Developer Moves AI Agents to a Dedicated Linux Workstation Using Incus System Containers — kevinkern · 2026-10-21
- Nvidia guides 70% growth citing unmet AI demand, while media panics over a $500M Korean neocloud IPO — firstadopter · 2026-10-21
- Pichai: everyone is massively short on AI compute — 'everyone wishes you had two times what you had planned' — firstadopter · 2026-10-21
- Stanford CS336 Lecture 6: fused Triton kernels cut softmax memory traffic ~4x — stanfordnlp · 2026-10-21
- General Compute partners with Positron AI: Atlas memory-centric hardware ships Q1 2027 — aftahi_ai · 2026-10-21
- AI bond issuance hits $419B in 2026, up 90% from all of 2025 — SumitGup · 2026-10-21