HarnessEvolve treats agent self-improvement like debugging, hitting 86.9% accuracy
rohanpaul_ai · x · 2026-09-04
Self-improving agents have a core problem: when a long run fails, they often can't tell which step caused it. A new arXiv paper, HarnessEvolve, treats agent self-improvement like software debugging — locate where a failed run first went off track, fix the recurring cause, then reject any edit that breaks existing behavior.
Key details and results:
- Clusters errors into recurring patterns and can edit the whole agent harness: prompts, skills, tools, scripts, and execution logic
- On CloudCoreNetwork-QA with Qwen3.6-27B, full HarnessEvolve reached 86.9% accuracy; removing reference trajectories dropped it to 57.8%
- Beat the strongest baseline by 21.6 percentage points
- Candidate edits pass gates for training-data leakage, prompt bloat, regressions on recent batches, and held-out validation
Related event: HarnessEvolve: Debugging Self-Improving Agents Like Code(2 posts)→
More from coding & agent
- Foundry Toolkit tutorial series wraps up with a career multi-agent system built on MCP — AmyKateNicho · 2026-09-04
- Microsoft names Project Zenith: dev-focused Windows shipping with AMD Ryzen AI Halo chips — tomwarren · 2026-09-04
- Claude Code user finds Codex CLI lacks context, limit and subagent visibility — MaxLenormand · 2026-09-04
- AI agents in Slack are assistants — but should they be workflow participants with identities? — Innowise_ · 2026-09-04
- AI debugging agent surfaces elusive bug missed across multiple sessions — antoine_chaffin · 2026-09-04
- Turning agent traces into training data: capture, sampling, and labelling, worked through — spilldahill · 2026-09-04