Self-Harness Lets Agents Modify Their Own Harness
量子位 · wechat · 2026-07-18
The Shanghai Artificial Intelligence Laboratory proposed Self-Harness: a method allowing models to automatically mine failure modes from their harness based on execution trajectories, propose constrained modifications, and decide whether to adopt them via regression testing.
The core workflow has three steps:
- WeaknessMining: Induces reusable weaknesses from failure trajectories, verifier feedback, and behavioral causal relationships;
- HarnessProposal: Generates candidate modifications within a limited editable range and explains potential regression risks;
- ProposalValidation: Uses held-in / held-out regression testing to filter, keeping only changes that deliver actual improvements without significant degradation.
The paper evaluates this on Terminal-Bench-2.0, keeping the model, tool environment, and evaluation protocol fixed while only altering the outer Harness. All three backends saw improvements:
- Qwen3.5-35B-A3B improved by 104%
- MiniMaxM2.5 improved by 28%
- GLM-5 improved by 24%
The authors observed that different models have distinct weaknesses: some delay output generation, others get stuck in loops after tool failures, and some need tighter control over shell state transitions. This proves Self-Harness isn't just adding a generic prompt, but integrating the model into an engineering loop of "finding issues, proposing fixes, and validating them."
Related event: Agent Harness Self-Improvement and Domain-Specific Design(6 posts)→
More from coding & agent
- Chaining dependent MCP tool calls: no rollback, duplicate risk — agentrsdg · 2026-09-11
- DeepMind-led paper makes design docs the source of truth, code disposable — SMART regenerates in 1.5-3h for ~$100 — Roger_M_Taylor · 2026-09-11
- Agent-built classifier labels 192k docs for $0.70 vs $13-26 with frontier LLMs — vanstriendaniel · 2026-09-11
- MathModelAgent gains traction: auto-solves math modeling and writes a submission-ready paper — jihe520 · 2026-09-11
- alphaXiv open-sources OpenResearch to run parallel research agents with any model — alphaXiv · 2026-09-11
- DeskcommCRM: open-source AI sales CRM with native agents and WhatsApp hits 1k stars — melgarafael · 2026-09-11