Why Did Token Price Collapse Not Save Money? Developer Urges Focus on 'Cost per Completed Step'
truecakesnake · reddit · 2026-08-04
An AI model developer points out that despite the massive drop in cost per million tokens this year, actual business spending hasn't decreased.
- Core Pain Point: Cheaper models often require re-runs. They might stop early, output literal paths instead of globs, or burn output budget over-thinking, causing the orchestrator to retry. Three cheap failed attempts are often worse in wall-clock time and overall cost than one expensive successful attempt.
- New Metric Proposal: The traditional $/M (cost per million tokens) is misleading; the industry should shift focus to cost per completed step. This metric depends heavily on the engineering harness rather than just the model's sticker price.
- Context: The author discovered this while cycling through sparse-MoE tier models (including Ling-3.0-flash, which they work on) and wants to know the spread across models when measured this way.
More from AGI Musings
- Expert Predicts Longevity Escape Velocity is Unlikely, Sets Median Lifespan Goal of 110 — MaxUnfried · 2026-08-04
- AMD Faces AI Earnings Test as Open Models Threaten Closed Labs' Margins — AccBalanced · 2026-08-04
- OpenAI Alignment Team Under 2% of Staff, Ex-Researchers Urge Pivot to Safety — Miles_Brundage · 2026-08-04
- Zuckerberg: Every Business Will Have Its Own Customized AI in the Future — rohanpaul_ai · 2026-08-04
- Andrew Yang: AI's Impact on Work is a Step Change, Not Linear — Olivier__OG · 2026-08-04
- LLMs Show High Intelligence Without Consciousness: Are They Separate? — GateSpiritual5717 · 2026-08-04