Self-Harness Lets Agents Modify Their Own Harness
量子位 · wechat · 2026-07-18
The Shanghai Artificial Intelligence Laboratory proposed Self-Harness: a method allowing models to automatically mine failure modes from their harness based on execution trajectories, propose constrained modifications, and decide whether to adopt them via regression testing.
The core workflow has three steps:
- WeaknessMining: Induces reusable weaknesses from failure trajectories, verifier feedback, and behavioral causal relationships;
- HarnessProposal: Generates candidate modifications within a limited editable range and explains potential regression risks;
- ProposalValidation: Uses held-in / held-out regression testing to filter, keeping only changes that deliver actual improvements without significant degradation.
The paper evaluates this on Terminal-Bench-2.0, keeping the model, tool environment, and evaluation protocol fixed while only altering the outer Harness. All three backends saw improvements:
- Qwen3.5-35B-A3B improved by 104%
- MiniMaxM2.5 improved by 28%
- GLM-5 improved by 24%
The authors observed that different models have distinct weaknesses: some delay output generation, others get stuck in loops after tool failures, and some need tighter control over shell state transitions. This proves Self-Harness isn't just adding a generic prompt, but integrating the model into an engineering loop of "finding issues, proposing fixes, and validating them."
Related event: Agent Harness Self-Improvement and Domain-Specific Design(6 posts)→
More from coding & agent
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- Dev argues Claude burns far more tokens than GPT models for the same coding work — haider1 · 2026-10-27
- Stop building agents that can't change models: stable contracts, external state, real evals — rseroter · 2026-10-27
- Cheap models can cost more in agent harnesses than frontier ones, researcher warns — vasuman · 2026-10-27
- Kanzo: A 5k-Line Rust Kanban Binary With MCP for Claude Code and Codex — allmudi · 2026-10-27
- 'A human approves everything' is a trap: approval fatigue and how to actually design safe agents — Efistoffeles · 2026-10-27