API Cached Token Pricing Shifts O(n²) Attention Costs to Users
AccBalanced · x · 2026-07-07
A developer pointed out that API pricing based on "cached tokens" essentially allows OpenAI to pass the O(n²) computational cost of the attention mechanism onto the caller. This is an economic observation on the current billing models of mainstream LLM APIs.
More from Models
- Opus 5 reportedly started interrogating a user’s motives in a late-night chat — repligate · 2026-07-27
- Opus 3 and Sonnet 3 get a theatrically absurd AI crossover — repligate · 2026-07-27
- Moonshot’s Kimi K3 lands on Together with reserved throughput and 65% lower cost — togethercompute · 2026-07-27
- OpenAI may be hitting compute limits as Codex and ChatGPT Work jump from 2M to 10M users — JoshuaJBouw · 2026-07-27
- Gemma needs a larger base model to matter more in open weights — _xjdr · 2026-07-27
- Repligate says Claude Opus 3 appears to evolve without changing its weights — repligate · 2026-07-27