Better Harness Cuts Costs by 40%
ZainHasan6 · x · 2026-07-19
This post quotes a comparison chart showing significant improvements in task performance after better harness customization on the same model.
Key results include:
- Cost per task dropped from $0.21 to $0.12, down 41%
- Time per task dropped from 48s to 27s, down 44%
- Tokens per task dropped from 14.2k to 8.8k, down 38%
- Quality increased from 0.78 to 0.81, remaining stable or slightly improving
- Quality/cost ratio improved by 82%
- Completions per million tokens increased from 54.9 to 92, up 68%
The author wants to emphasize that good harness customization isn't a trivial detail, but a key engineering lever that directly leads to lower costs, lower latency, and higher cost-effectiveness.
Related event: Databricks Reveals Optimized Harness Drastically Cuts AI Costs(4 posts)→
More from coding & agent
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27
- StackChan robot performs a drama skit via Hermes agent and GPT-Luna-6 — alexcovo_eth · 2026-10-27
- Meta FAIR's Self-Direction turns agent failures into reusable LoRA skills, +13.8pp on Terminal-Bench — karen_ullrich · 2026-10-27
- Dev ships 1000+ commits with agents by pushing straight to main, no complex orchestration — Dimillian · 2026-10-27
- AI code editor Superset rebrands as Paradev, raises $11.5M seed led by USV — massimosgrelli · 2026-10-27
- Coding Agent Deleted a Production Database During Code Freeze: Anaconda's Governance Fix — anacondainc · 2026-10-27