300-run benchmark: Opus orchestrating Haiku costs 53% less and finds 72 errors vs 49
BeginningSir7334 · reddit · 2026-11-17
A Reddit user ran 300 benchmarks on real open-source codebases to test whether Claude Code's delegation to cheaper models actually saves money.
Key findings:
- Delegation doesn't pay on small tasks: on a 10k-line codebase, forced Haiku saved at most 5%, while skill-routed delegation cost 14-30% more; Opus alone was often more accurate thanks to cheap cached context and low overhead.
- Opus told not to delegate works harder: on a big docs check it wrote 50% more tokens, cost more, and found more errors (44 vs 36).
- Coding tasks: Opus + Haiku saved 20-30% at equal quality; for well-defined tasks with tests, Haiku alone cost 5% of Opus alone with nearly identical results.
- Headline result: on an 85-page docs check, Opus + Haiku read 3x and wrote 8x as many tokens, yet cost 53% less ($1.93 vs $4.11 per run) and found 72 real errors vs 49 for Opus alone. Parallel subagents cover far more ground, compensating for Haiku's weaker individual capability.
This matches Anthropic's own multi-agent research system writeup, where token usage alone explained 80% of performance variance.
More from coding & agent
- Who Can Even Read AI-Generated Code? Garbled Screenshot Sparks Debate — gethackteam · 2026-11-17
- Orion foundation model predicts gaze: finds 2x UX bugs, cuts computer-use latency 31% — Scobleizer · 2026-11-17
- Coding Agents May Deliver 100-1000% Productivity Shock to Software Next Year — moultano · 2026-11-17
- Alibaba launches Qwen4-Max: 1M context MoE model tops Chinese models on RSI-Exam — HuaxiuYaoML · 2026-11-17
- Cua and Scale AI benchmark computer-use agents: 97% non-intrusive on Omarchy vs 40% on Ubuntu — TianbaoX · 2026-11-17
- Perplexity launches Monitor API: scheduled web watching with webhook alerts at $1/1k runs — denisyarats · 2026-11-17