LLM cost curves questioned: distillation gains vs the $0.01/mtok price floor
davidmanheim · x · 2026-09-23
David Manheim challenges a widely-shared LLM capability/cost chart, arguing it wrongly merges capability gains with cost declines. Distilled small models do offer slightly lower quality at vastly reduced costs, partially supporting the claim — but a structural price floor remains: at $0.01/mtok just to serve tokens over web infrastructure, third/fourth-tier models can't differentiate on price since no one bothers serving the cheapest options.
Related event: Scholars Challenge Viral Chart Claiming 100,000x Drop in AI Prices(10 posts)→
More from Infra
- NVIDIA reportedly cancels RTX 5080 24GB over GDDR7 supply and cost — ceciletamura · 2026-11-11
- Rivet launches SQLite Forking: 20ms agent-thread forks with zero data copying — samgoodwin89 · 2026-11-11
- French netizen proposes datacenters pay local residents compensation — clemnt · 2026-11-11
- GPU Rental Prices Hit Record Highs: B200 at $6.97/hr, H100 at $3.06 — inductionheads · 2026-11-11
- A Datacenter in Every Office: Building Work-Safe Local AI Racks — dee_hw · 2026-11-11
- "Compute is about power": the AI race's real bottleneck is energy, not chips — _sholtodouglas · 2026-11-11