Berkeley study reveals the 'harness tax': harness choice impacts coding agent cost more than accuracy
arena · x · 2026-09-29
LMArena highlights new research by Melissa Pan, PhD candidate at UC Berkeley's Sky Computing Lab, benchmarking Claude Code vs Codex vs Pi.
She introduces the hidden "harness tax": the system surrounding an AI model materially changes its cost and performance. Among three surprising findings: harness choice impacts cost more than accuracy — meaning picking the right harness matters as much as the model itself when building coding agents on realistic budgets.
Related event: Berkeley Study: Coding Agent Harness Choice Drives Cost More Than Accuracy(2 posts)→
More from coding & agent
- Generic API-catalog MCP server vs one tool per job: a context-bloat experiment — MandeepKaur9407 · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- Effective HTML: 4.2k-star agent skills turn wireframes into self-contained interactive prototypes — tom_doerr · 2026-11-16
- skills-gate: one gate skill fronts 148 agent skills at just 389 tokens — Distinct-Pie2389 · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- shadcn boosts @uisdk: one-shot UI rebuild that's instantly E2E-testable by AI — shadcn · 2026-11-16