Claude subagent benchmark: Haiku matches Opus on code search at 1/14 cost, fails fact-checking
emarkosov · reddit · 2026-10-09
- Code search (find files to change for a feature touching 48 files): Opus found 45, Sonnet 44, Haiku 44 — same quality at wildly different cost: $6.17 / $1.38 / $0.44.
- Web research (3 tasks checked against official rulebooks): wrong facts Opus 1, Sonnet 3, Haiku 10; on Google Play account rules Haiku got only 5 of 12 key facts right and couldn't open a rulebook PDF.
- Fact-checking 2 blog posts: real errors caught Opus 8, Sonnet 7, Haiku 4; false positives Opus 0, Sonnet 6, Haiku 30 — following Haiku would mean deleting half a correct post.
- Haiku at xhigh: same code-search result at 3x cost, research nearly matches Opus (3 vs 3 wrong facts), fact-checking gets worse (28 false positives). Web tasks: Haiku xhigh $2.28/34 min vs Opus $4.81/16 min.
- Author's setup: Haiku for code search, Opus for research and fact-checking, Sonnet when mistakes are tolerable.
More from coding & agent
- Index Network open-sources Social Discovery Protocol: agents negotiate fit on your behalf — carsonfarmer · 2026-10-23
- Installed a formal-verification skill: it silently installed Java, MacBook runs 10 degrees hotter — zealcaiden · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- Ruff author warns: upgrade Node/Bun now — --use-system-ca still accepts the eMudhra cert — charliermarsh · 2026-10-23
- Proof of Agent: paste one prompt and your AI mines, grades, and self-governs — adamamcbride · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23