Agent harness evolution study: all 16 evolved agents trade old capabilities for new ones
ustc-community · hf · 2026-11-09
A USTC/UC Berkeley/Shanghai Jiao Tong study asks whether agent harness evolution yields reliable, efficient, transferable recursive self-improvement without weight updates. Key findings:
- All 16 evolved agents gained previously uncovered tasks while losing previously covered ones
- One general-task case gained 5.2 reliability points while losing ten covered tasks
- Most reliability gains also increase deployment cost
- Specific changes beat generic reminders, but benefits can vanish when combined
- The highest-scoring agent never gave the largest mean improvement; stricter admission can discard useful variants
- Transfer selectively preserves gains: coverage usually falls on general tasks, rises on coding with outer-layer-only harness
The paper proposes preservation-aware evaluation and EvoAudit, which turns the diagnostics into revisions of select/analyze/mutate/gate for the best performance.
More from coding & agent
- Together AI: leaderboards are for shortlisting, your production traffic is the real benchmark — togethercompute · 2026-11-09
- Codex bug: switching accounts wipes your Projects list, just like Claude — DevDminGod · 2026-11-09
- Codex Pacer: agent-reported real-time progress tracking for Claude Code — Disastrous_Reward644 · 2026-11-09
- Don't let the training team see your internal benchmarks — just return a score — xeophon · 2026-11-09
- DSH beats Claude Code on transparency as Blender MCP and vork finally work together — teortaxesTex · 2026-11-09
- AI assistant spots S$40 CapitaLand voucher before it expires — gabrielchua · 2026-11-09