Study of 7 models across Claude Code, Codex, Pi: harness barely affects success but swings cost
DavideCrapis · x · 2026-09-17
Melissa Pan's team evaluated 7 models across the Claude Code, Codex, and Pi harnesses and found three surprising results: (1) harness choice has little effect on task success rate but can significantly affect cost; (2) a simple harness can be competitive; (3) the model's native harness isn't always the best. The study addresses a long-standing gap — millions use coding agents, yet the impact of harness choice remained unclear.
Related event: 7 Models × 3 Coding Harnesses: Harness Choice Drives Cost, Not Success(7 posts)→
More from coding & agent
- Tencent unveils Hyra 2.0, a research agent that builds reusable scientific tools — TencentHunyuan · 2026-11-02
- What broke when I ran a leaderless agent swarm with no orchestrator — 7 failure modes and fixes — LostHumanity67 · 2026-11-02
- Forced yes/no prompts push GPT and Claude to 95%+ confidence on Goldbach's unproved halt test — Nir777 · 2026-11-02
- Automation took over my daily work — so why am I still sitting in the office for 8 hours? — Am1rSori · 2026-11-02
- General Reasoning explains how to train agent swarms with RL and latency pressure — rosstaylor90 · 2026-11-02
- "Coding isn't the bottleneck anymore — usage windows are," says founder Kyle Gawley — kylegawley · 2026-11-02