KV cache economics: why a $200 Claude plan can hand out $14,000 of API-equivalent tokens and still profit
rachittshah · x · 2026-11-21
Gergely Orosz explains why inference is so profitable: KV cache reads are very cheap and speculative decoding makes sessions efficient, and coding workloads lean heavily on both.
- Claude/ChatGPT Max plans offer 10-70x the usage of API billing — a $200/month plan can yield up to $8,000-14,000/month of API-equivalent tokens (SemiAnalysis), yet stays profitable because 90%+ of a coding agent's tokens are cache reads whose true serving cost is a small fraction of list price.
- Typesafe CEO Diogo Almieda: more tool calls → more cache reads → more profit, so agentic workloads keep pushing labs' inference margins up.
- The debate started when Vlad Mihalcea noted his $200/year Pro plan covered 16B tokens, $100,000 at API rates.
- Implication: self-hosting open-weight models makes sense for agents at scale; Fable 5.1 cutting cache read pricing from $1.00 to $0.25 per million tokens suggests prices will keep falling.
More from Infra
- Self-hosted cloud GPUs beat Higgsfield and Runway APIs on cost and flexibility — Al_Grigor · 2026-11-21
- NVIDIA GTC DC lineup: Ian Buck, Catanzaro and Lebaredian sessions on infra, agents, physical AI — nvidia · 2026-11-21
- Baseten breaks down its security threat model as agent swarms accelerate attacks — baseten · 2026-11-21
- Nvidia's rivals are vying to be the 'Central Bank' of AI, says investor — GavinSBaker · 2026-11-21
- Benchmarking 11 open models' post-GQA attention: up to 139x less memory at 128K — bariskasikci · 2026-11-21
- Anthropic files for Nasdaq IPO — why you should keep a local open-source LLM as backup — alex_verem · 2026-11-21