30-run test: Opus costs 16x more than Haiku for small bug fixes with identical results
Sufficient-Storage87 · reddit · 2026-10-15
Following up on a routing post, the author benchmarked 5 Claude Code setups on real bugs from tomlkit and cachetools — 30 runs total, all passing hidden tests, repriced at API list prices from transcript tokens.
- Haiku alone: $0.013/fix (48s)
- Haiku + Opus advisor: $0.062
- Sonnet alone: $0.079 (21s, fastest)
- Opus + Haiku subagent: $0.126
- Opus alone: $0.206
For small, well-defined fixes Opus buys essentially nothing at 16x the cost. A previously tested harder bug showed a smaller gap with Sonnet winning. Caveat: tomlkit may be in training data.
More from coding & agent
- Index Network open-sources Social Discovery Protocol: agents negotiate fit on your behalf — carsonfarmer · 2026-10-23
- Installed a formal-verification skill: it silently installed Java, MacBook runs 10 degrees hotter — zealcaiden · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- Ruff author warns: upgrade Node/Bun now — --use-system-ca still accepts the eMudhra cert — charliermarsh · 2026-10-23
- Proof of Agent: paste one prompt and your AI mines, grades, and self-governs — adamamcbride · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23