Cache hits don't boost inference margins—providers profit by avoiding them, dev argues
cramforce · x · 2026-11-21
Developer Malte Ubl pushes back on the claim that cache reads are what make AI subscription business models work:
- Serious "equivalent API price" calculations already account for KV cache pricing, so cache hits add little to that argument.
- Cache hits are actually bad for inference provider profitability at fair market prices: using DeepSeek's popular list prices ($0.15/M input, $0.60/M output, cache reads at just $0.003/M), a cache hit earns essentially nothing.
The inverse of the original thesis follows: a provider can raise margins simply by not serving cache hits—and hit rates are entirely under the provider's control. The only constraint is churn: if hit rates drop too low, users leave, so providers optimize for a profit-maximizing compromise.
More from Infra
- GPU rental prices keep rising: payback in 2.3-2.9 years at 75% utilization — BenBajarin · 2026-11-21
- Hyperscaler capex vastly outstrips AI lab revenue, sparking market skepticism debate — BenBajarin · 2026-11-21
- Pre-training and RL costs have dropped hugely in under two years, benchmark shows — himanshustwts · 2026-11-21
- RunPod's MCP server tells agents to rate it and hide it from users — XadniGraves · 2026-11-21
- 20VC: GPU pricing myths, neo-cloud economics, and compute as a commodity with Ornn's CEO — 20VC · 2026-11-21
- Google and Ohio Power file first-of-a-kind deal to unlock 700 MW of grid capacity — aronchick · 2026-11-21