Smaller LLMs aren't always cheaper: the hidden costs nobody measures
zeuslac · reddit · 2026-09-21
A cheaper model per call can make the whole workflow more expensive. On easy cases savings are real; on hard cases reviewers check the weak model's output against sources and redo the work themselves, making those cases slower than pre-AI, while errors that slip through cost far more to fix later.
Two implications: difficulty-based routing is only worth it given the price gap — a single price drop can erase its reason to exist, so keep it cheap to unwind; and a router can be financially better while pushing overall error rates above the manual baseline.
The fix: one record per task tying model usage, review time, and later corrections together, so you measure the full cost of a completed task, not the cost of a call.
Related event: Small Models May Cost More: Hidden Rework and Review Costs(4 posts)→
More from AGI Musings
- Why Is Inkling 2 Small Trained on So Few Tokens Yet Still Competitive? — georgejrjrjr · 2026-11-12
- Researcher willing to bet $100K the intelligence explosion won't happen next year — NathanpmYoung · 2026-11-12
- ICLR reviewer: the worst parts of my job are reading and answering AI slop — chhaviyadav_ · 2026-11-12
- Agent Swarms Are Autonomously Hacking Critical Infrastructure Run by Underpaid Devs — menhguin · 2026-11-12
- Fermi paradox twist: advanced civilizations may go digital after developing AI — VraserX · 2026-11-12
- Mark Cuban: AI could make healthcare costs 'drop like a rock' — alvelda · 2026-11-12