Harness Choice Barely Affects Success Rate but Can Quintuple Coding Agent Costs
UC Berkeley's HarnessTax study tested 21 model-harness combinations across 60 tasks and found that while success rates remain similar, costs can differ up to 5x depending on the harness. The findings suggest harness selection is a key lever for cutting coding agent costs.
2026-09-21 ~ 2026-09-22 · 3 related posts
- Episode 1: 7 Models × 3 Coding Harnesses: Harness Choice Drives Cost, Not Success(2026-09-17, 7 posts)
- Episode 2: Benchmark of 21 model-harness combos: agent framework changes cost, not success rate(2026-09-19, 2 posts)
- Episode 3: Harness Choice Barely Affects Success Rate but Can Quintuple Coding Agent Costs(2026-09-21, 3 posts)
- Arena: coding-agent harnesses show up to 5x cost differences at similar success rates — thione · 2026-09-21
- Same model bills 5x more in a different harness: 21 combos tested across 60 tasks — CShorten30 · 2026-09-22
- UC Berkeley's HarnessTax study: agent harness barely changes success rate but costs vary 5x — solyarisoftware · 2026-09-22