LLM cost curves questioned: distillation gains vs the $0.01/mtok price floor
davidmanheim · x · 2026-09-23
David Manheim challenges a widely-shared LLM capability/cost chart, arguing it wrongly merges capability gains with cost declines. Distilled small models do offer slightly lower quality at vastly reduced costs, partially supporting the claim — but a structural price floor remains: at $0.01/mtok just to serve tokens over web infrastructure, third/fourth-tier models can't differentiate on price since no one bothers serving the cheapest options.
Related event: Toby Ord and Others Challenge Viral "AI Prices Fell 100,000x" Chart(5 posts)→
More from Infra
- Dreamscale serves NVIDIA's DreamZero robot model 8% faster with half the silicon — chris_j_paxton · 2026-09-23
- Agents Make Data Stacks Unforgiving: Stale Data Can Trigger Wrong Orders — bigdata · 2026-09-23
- Silent LLM Cost Errors: Inconsistent Token Accounting Undercounts Usage by Up to 46x — qaiser_mehdi · 2026-09-23
- Redis ships Radar, Search on Flex and agent memory tools as 97% back context engineering — antirez · 2026-09-23
- 24GB AMD GPU Struggles With Local Video Models: 2 Minutes for 2-Second 480p — OneMoreName1 · 2026-09-23
- ByteDance accounted for nearly 75% of Nscale's 2025 sales, used Norway site to access Nvidia chips — nathanbenaich · 2026-09-23