Why DeepSeek's API Is So Cheap: Tiny Model Size Boosts Single-Chip Throughput
AravSrinivas · x · 2026-08-03
Addressing questions about DeepSeek's extremely low API pricing, the author points out that their core advantage lies in a ridiculously small model size. Compared to Anthropic's Opus and Sonnet, DeepSeek's model is 5x to 10x smaller, allowing what used to require an 8-chip node to run on a single chip.
Because it's so small, the model runs extremely fast and can multi-serve multiple users on a single chip while maintaining decent speeds. By the author's math, DeepSeek can handle roughly 40x more traffic than Opus using the exact same compute. This high inference efficiency is why they can stay profitable even at crazy low prices.
Related event: DeepSeek's Low-Priced API Profitability Driven by Ultra-Small Model(2 posts)→
More from Infra
- Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training — BaiduResearch · 2026-10-16
- GYRO quant of Qwen3.8-Flash-Next tested: near Q3 quality at much smaller size — PlasticRevenue4601 · 2026-10-16
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16