Legora's FrontierHarness eval: Codex tops at 66.7%, harness cost spreads 17x
chetanp · x · 2026-09-03
Legora's FrontierHarness benchmark compares 12 coding agent harnesses on identical tasks, showing harness design matters far more than model choice in vertical AI.
- Quality leader: Codex, 66.7% pass rate at $3.47/task
- Balanced pick: Pi, 60.0% at just $2.43/task
- Cost leader: Exo Harness at $1.05/task (53.3%)
- Speed leader: DSH Minimal, 5m41s median runtime
- Claude Code ranks #2 on pass rate (63.3%) but costs $18.34/task median — over 5x Codex
- On median cost per successful task, OpenCode is cheapest ($0.0615); cache hit rate tops out at 88% (Codex, Kimi Code)
Takeaway: harness engineering is a hugely underrated lever for performance and cost.
Related event: FrontierHarness: Same Model, 17-Point Gap Across Agent Harnesses(2 posts)→
More from coding & agent
- Anthropic borrows lawyers' ethics for Claude agents that closed 186 deals with no human approval — rohanpaul_ai · 2026-10-18
- One prompt turns Claude Code into a full motion-graphics video pipeline with Remotion — aziz4ai · 2026-10-18
- Prompt injection via email can hijack your AI agent — jev-guard adds risk-scored guards — sujingshen · 2026-10-18
- 14-Agent AI Film Pipeline on One RTX 5090: Fixing Cut Defects Between Clips — developervkmp · 2026-10-18
- Meta paper introduces "agent plasticity": best-performing agent isn't the most efficient learner — rohanpaul_ai · 2026-10-18
- Dev ships a bushfire prediction platform in a weekend with frontier models — CtrlAltDwayne · 2026-10-18