7 models across Claude Code, Codex, Pi: harness choice drives cost, not success
matei_zaharia · x · 2026-09-17
Melissa Pan's team evaluated 7 models on Claude Code, Codex, and Pi coding-agent harnesses, with three surprising findings: (1) harness choice has little effect on task success rate but significantly affects cost; (2) a simple harness can be competitive; (3) the native harness isn't always the best. Millions use coding agents, yet harness impact was previously unclear — full details in the thread.
Related event: 7 Models × 3 Coding Harnesses: Harness Choice Drives Cost, Not Success(7 posts)→
More from coding & agent
- Auditing my MCP credential broker: the worst hole was the config file — Relevant_Mousse_9117 · 2026-11-02
- AWS's open-source Strands Harness claims 45% cheaper than Claude Code, reigniting harness debate — blaizedsouza · 2026-11-02
- Xiaomi MiMo's CodeSpecWeave trains coding agents on interdependent specs for long-horizon coding — XiaomiMiMo · 2026-11-02
- Sketchy v0.1.6: AI diagrams use 29% fewer output tokens than plain text — Nkgsporty · 2026-11-02
- Avoiding AI brain rot: Miles Cranmer's lecture on learning with AI — _ScottCondron · 2026-11-02
- Scrapling web scraping framework tops 92k GitHub stars, ships MCP server for AI workflows — D4Vinci · 2026-11-02