Benchmark of 21 model-harness combos: agent framework changes cost, not success rate

A viral study benchmarked 7 models across 3 coding harnesses (21 combos), finding that the choice of agent framework barely affects success rates but dramatically changes cost.

2026-09-19 ~ 2026-09-20 · 2 related posts

• 21 model-harness pairs tested: framework choice barely moves success rate but swings cost — iScienceLuvr · 2026-09-19
• Evaluating 7 Models Across Claude Code, Codex, and Pi: Harness Choice Drives Cost, Not Success — CShorten30 · 2026-09-20