Study quantifies the 'Harness Tax': lean 4-tool setup matches Claude Code and Codex
bendee983 · x · 2026-09-22
A new study measures the "Harness Tax" — the cost/performance gap between harnesses with task and model held constant. Key findings:
- Complex harnesses like Claude Code move cost-per-task far more than aggregate success
- A minimal harness (Pi, 4 tools) holds the frontier on both benchmarks vs. Codex and Claude Code
- Models don't necessarily perform best in their own vendor's harness — 9 of 12 cases favored a non-default split
Advice: start lean and add weight only when failures or needed features earn it. Author interviewed co-author Melissa Pan for Alpha Signal.
More from coding & agent
- Ten wild Gemini 4 Carbon demos in 24 hours: CUDA meadows, V8 engines, mech stomps — socialwithaayan · 2026-11-12
- How to cut Fable 5.2 token costs by 90%: Arc routing plus a lean CLAUDE.md — EXM7777 · 2026-11-12
- Hamel Husain's complete AI evals framework for PMs and engineers — aakashgupta · 2026-11-12
- AI coding in one year: from unusable to full automation with no code review — angelaqdai · 2026-11-12
- Interactions Tool Searches Web Animations by Motion for Claude Code to Rebuild — yuwen_lu_ · 2026-11-12
- Agent Substrate Calculator Estimates Session Resume Speed Under Load — pjausovec · 2026-11-12