Higher Reasoning Effort Costs Up to 42x and Can Break Tasks Lower Effort Solves
togethercomputer · hf · 2026-10-28
Together AI with Johns Hopkins and Cornell measured the counterfactual effect of reasoning effort levels across proprietary and open-weight models on four benchmarks, framing reasoning effort as a credence good — you pay up front and can't assess the marginal value afterward.
- Higher effort raises aggregate accuracy but frequently fails tasks that lower effort solved.
- On easy tasks all levels solve, the highest effort level costs up to 42x the lowest.
- Effort changes not just token count but how agents work: more self-verification and searching instead of direct progress, lengthening trajectories without changing outcomes.
- Practical takeaway: start at low effort and escalate on demand — a directly usable cost-optimization strategy for API users and agent engineers.
More from coding & agent
- Early users praise DeckWerk after weeks of using it for real talks — vincesitzmann · 2026-10-28
- Agentic video editing goes pro: senior editors 5x output, costs to collapse — dotey · 2026-10-28
- Software is going agent-first: sites, browsers, even a new Rust-like language built for agents — mobileraj · 2026-10-28
- HABIT-Bench: Best agent memory system scores just 33.61% at inferring user habits — SJTU · 2026-10-28
- Regehr's 'Piano Test': would you trust formally verified code with your life? — moyix · 2026-10-28
- Vibe-coded 'AI 浮生记' game satirizes AI startup life with compute black-market trading — vista8 · 2026-10-28