DeepSeek's 100 PiB SSD cache tier analyzed: ~90% serving margin at $38/card-hour implied
teortaxesTex · x · 2026-10-20
A technical deep-dive shared by teortaxesTex reveals DeepSeek's KVCache storage architecture and margin math:
KVCache storage
- 3FS provides 100 PiB of SSD storage for KVCache
- TTL of 3 days for agent traffic, 7 days for chatbot; the 3-day TTL only costs chatbot hit rate 0.5-1%
- Optimized for 112 KiB block granularity: peak aggregate read bandwidth 10 TiB/s, writes 850 GiB/s
- Paged Block KVCache lives on a dedicated 3FS cluster, decoupling storage from compute and enabling the 96% cache-hit rate
Margin estimate (an assumption, not a DeepSeek disclosure)
- At $38 revenue per average card-hour, $3-5/card-hour all-in serving cost implies 87-92% gross margin; a central $4/hour assumption yields 90%
- 90% margin requires total serving cost below $3.80/card-hour; 85% below $5.70; 80% below $7.60
- Reference point: SemiAnalysis's public model quotes $2.31/chip-hour all-in for GB300 NVL72
- These are inference delivery margins only, excluding training, research, and corporate expenses
More from Infra
- Together, Fireworks, Prime Intellect and Modal are all converging on inference + RL training — NielsRogge · 2026-10-20
- NVIDIA GTC kicks off in Berlin with Jensen Huang keynote set for tomorrow — NVIDIAAI · 2026-10-20
- Hyperdrives CEO interview: cheaper power-dense electric motors via cooling breakthrough — MarvinTBaumann · 2026-10-20
- MiniMax H3 video generation runs on a single RTX 3060 12GB via GGUF quantization — Ok-Inevitable8391 · 2026-10-20
- ByteDance, Alibaba, Tencent place over $35B in Nvidia chip orders amid supply crunch — pstAsiatech · 2026-10-20
- DeepSeek-V4.1-Flash tops Agent Arena open models, ranks #11 overall at $0.20/task — vista8 · 2026-10-20