Evaluating 7 Models Across Claude Code, Codex, and Pi: Harness Choice Drives Cost, Not Success
CShorten30 · x · 2026-09-20
A coding agent evaluation (trending on Hacker News as HarnessTax) benchmarks 7 models across Claude Code, Codex, and Pi, yielding three surprising findings:
- Harness choice has little effect on task success rate, but significantly affects cost — a wrong harness burns money without hurting quality;
- A simple harness can be competitive, so elaborate orchestration may not pay off;
- The model's native harness isn't always the best option.
Practical takeaway for developers: validate simple harnesses on cost/benefit before stacking complex agent frameworks.
More from coding & agent
- Ten wild Gemini 4 Carbon demos in 24 hours: CUDA meadows, V8 engines, mech stomps — socialwithaayan · 2026-11-12
- How to cut Fable 5.2 token costs by 90%: Arc routing plus a lean CLAUDE.md — EXM7777 · 2026-11-12
- Hamel Husain's complete AI evals framework for PMs and engineers — aakashgupta · 2026-11-12
- AI coding in one year: from unusable to full automation with no code review — angelaqdai · 2026-11-12
- Interactions Tool Searches Web Animations by Motion for Claude Code to Rebuild — yuwen_lu_ · 2026-11-12
- Agent Substrate Calculator Estimates Session Resume Speed Under Load — pjausovec · 2026-11-12