Matt Shumer: The 'cheap' model often costs more once you count retries
mattshumer_ · x · 2026-11-21
Matt Shumer argues against the reflex of mandating cheaper models to save money: sticker price per token isn't the cost of a finished job. Smarter models take fewer steps and make fewer mistakes, so you buy far fewer tokens. He likens it to hiring a student at $20/hr for 100 hours versus Einstein at $1,000/hr for one. Concrete examples:
- Do-overs: a task Opus nails in one try may take Sonnet three feedback rounds; at half the per-token price that's 1.5x Opus's total cost, before counting human time.
- Effort settings: people pick the cheap model then crank effort to max to compensate, inflating inference spend.
His advice: compute end-to-end task cost (total tokens plus human time) rather than comparing per-token rates — and send that math to your boss before they downgrade your model access.
More from Venture
- Someone spun up 5 niche ranking sites to claim #1-#5 on outbid.lol — jonathan_wilke · 2026-11-21
- New York Times: Can Anthropic Revive the A.I. Trade? — nordicinst · 2026-11-21
- Anthropic files for Nasdaq IPO — why you should keep a local open-source LLM as backup — alex_verem · 2026-11-21
- Stocks head toward worst week in over a year as Anthropic's debut fails to stem sell-off — GaryMarcus · 2026-11-21
- The 'Maritime Renaissance': 151 AI and robotics startups mapped across 8 regional clusters — clemnt · 2026-11-21
- NVDA down 53% YTD, Redditor predicts cheap GPU glut by 2027 as bubble deflates — Markus2816 · 2026-11-21