Better Harness Cuts Costs by 40%
ZainHasan6 · x · 2026-07-19
This post quotes a comparison chart showing significant improvements in task performance after better harness customization on the same model.
Key results include:
- Cost per task dropped from $0.21 to $0.12, down 41%
- Time per task dropped from 48s to 27s, down 44%
- Tokens per task dropped from 14.2k to 8.8k, down 38%
- Quality increased from 0.78 to 0.81, remaining stable or slightly improving
- Quality/cost ratio improved by 82%
- Completions per million tokens increased from 54.9 to 92, up 68%
The author wants to emphasize that good harness customization isn't a trivial detail, but a key engineering lever that directly leads to lower costs, lower latency, and higher cost-effectiveness.
Related event: Databricks Reveals Optimized Harness Drastically Cuts AI Costs(4 posts)→
More from coding & agent
- The browser main thread is expensive: a practical guide to JavaScript and CSS animation cost — jh3yy · 2026-09-11
- Inspired by OpenAI's 10,000-agent run, dev open-sources a crowdsourced agent problem-solving platform — Benjaminsen · 2026-09-11
- Lucid: open-source Mac app keeps your laptop awake only while AI agents run — Pitiful_Hedgehog_600 · 2026-09-11
- banteg's snail project crowdsources AI agents to finish matching Snail Mail's 20 remaining functions — banteg · 2026-09-11
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11