HarnessFix studies 30 agent repos and lifts task completion by up to 18.4 points
青稞AI · wechat · 2026-07-25
HarnessFix finds and repairs agent-harness flaws instead of blaming the model
A team from the Institute of Software, Chinese Academy of Sciences proposes HarnessFix, a framework for diagnosing and repairing flaws in the AgentHarness layer — the execution, tool, memory, orchestration, observability, verification, and governance stack around an LLM agent.
- The paper studies 30 popular open-source LLM agent repositories and analyzes about 57,780 development records. It finds that 26,174 records (45.3%) are harness-related, and all 30 repos contain such fixes.
- The most common issues are in Lifecycle, Tooling, and Observability; 29 of 30 repos touch those layers.
- HarnessFix converts execution traces into HTIR (Harness-aware Trace Intermediate Representation), then performs root-cause analysis along data-flow and control-flow paths.
- It maps diagnoses to scoped repair operators so fixes stay bounded to the relevant harness artifacts, and then uses regression-aware verification before accepting a patch.
In evaluation on GAIA, SWE-Bench Verified, AppWorld, and Terminal-Bench 2.0 Verified, HarnessFix improves task completion rate by 6.3 to 18.4 points over the initial harness. Compared with the strongest automatic baseline, it still wins by 2.6 to 5.0 points, while using 63.5% to 100.5% fewer tokens offline.
The paper argues that many “model failures” are actually harness failures: missing parameter checks, swallowed errors, weak completion criteria, and poor observability. The key message is that agent reliability can often be improved by fixing the surrounding runtime, not just the prompt or the base model.
More from coding & agent
- Grok adds Netlify connector: build and deploy sites straight from chat — thisiskp_ · 2026-10-26
- Nemotron 3.5 Super VL tested on 300 real bills: reasoning lifts accuracy 91% to 98% — Arindam_1729 · 2026-10-26
- How a free audit plugin cut this agent's token overhead from 40K to 19.2K — tomcrawshaw01 · 2026-10-26
- Open-source local studio bundles image, video, music and AI movie maker offline — lelus78 · 2026-10-26
- LangChain and Tavily to co-host NYC event on production agents using live web data, Nov 12 — LangChain · 2026-10-26
- SentinelForge: evidence-driven AI software repair with human-in-the-loop verification — Byte_Forge_2811 · 2026-10-26