Berkeley PhD Candidate Compares Claude Code vs Codex vs Pi: Harness Choice Hits Cost More Than Accuracy
arena · x · 2026-09-29
Arena released a video in which Melissa Pan, a PhD candidate at UC Berkeley's Sky Computing Lab and former Arena intern, benchmarks three coding agents — Claude Code, Codex, and Pi. She introduces the concept of the hidden "harness tax": how the system scaffolding around a model affects its cost and performance.
She reports three surprising findings, the first being that harness choice impacts cost more than accuracy — the same model wrapped in different agent setups can differ far more in spend than in quality. The full video covers all three findings and their implications for building useful coding agents on realistic budgets.
Related event: Berkeley Study: Coding Agent Harness Choice Drives Cost More Than Accuracy(2 posts)→
More from coding & agent
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Every's Frontier team on Claude Fable 5.2: overnight agent jobs, cost tradeoffs, real builds — every · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16