Tokens spent explained 80% of performance on Anthropic's browsing benchmark
Nir777 · reddit · 2026-11-19
- Anthropic's multi-agent research system beat a single agent by 90% on its own eval, but the same post admits token spend alone explained 80% of performance differences on a browsing benchmark.
- An April paper gave a single agent and a team the same reasoning budget: the single agent matched or beat the team on multi-step reasoning, across three model families.
- The accompanying image contrasts bad splits (serial tool-dependent subtasks) with good ones (parallelizable research subtasks).
- Takeaway: many multi-agent wins may just be extra compute; the fair test is one agent at the same budget, which the author runs in their newsletter.
More from AGI Musings
- Dean Ball challenges AI's "silent consensus" on human primacy — deanwball · 2026-11-19
- Paul Christiano: no alignment approach has >50% chance of working — 'that is the plan' — trevposts · 2026-11-19
- David Deutsch: Musk's Childhood Trance Shows Why No Behavioural Test Can Detect AGI — Paimaamu · 2026-11-19
- If AI hits targets, move to a four-day workweek? — MikkoH · 2026-11-19
- Moral Machine Creator Iyad Rahwan Interviewed on How AIs Should Make Moral Decisions — iyadrahwan · 2026-11-19
- Nina Schick: the risk is not having SuperIntelligence — first felt in cyber — NinaDSchick · 2026-11-19