Berkeley-led paper: evolved agents gain 5.2 reliability points while losing 10 tasks they once solved
rohanpaul_ai · x · 2026-11-09
A UC Berkeley/USTC-led paper examines agent harness evolution—letting agents rewrite their own prompts, skills, and tools while model weights stay frozen. Running 4 evolution methods (incl. GEPA, HyperAgents) with GPT-5.4 across 2 harnesses and 2 benchmarks, they re-tested each evolved agent 8 times: all 16 gained and lost tasks, one raised reliability 5.2 points while losing 10 previously covered tasks, and 8 of 11 more-reliable agents cost more per run. Highest-scoring agents never yielded the largest mean improvement, and transfer selectively preserves gains. The authors advocate tracking lost tasks, cost per run, and unseen-task performance, and release EvoAudit (code: simple-agent-lab/RSIHub).
Related event: Study Finds Agent Self-Evolution Trades Old Skills for New Gains(3 posts)→
More from coding & agent
- Peer-reviewed paper applies truth maintenance systems to LLM memory belief revision — letandrewcook · 2026-11-09
- System One models are back: decision/extraction APIs cut insurance claim flow from 61.7s to 19.3s — max_gladysh · 2026-11-09
- User builds a 'chief of staff' Grok agent that runs projects and sends one weekly brief — RachelVT42 · 2026-11-09
- 10 Open-Source Tools for Building Trustworthy AI Apps, from Langfuse to TruLens — Shruti_0810 · 2026-11-09
- Shipping AI agents: attack it before release, read the traces, not just the answers — Plastic_Secret1303 · 2026-11-09
- Appwrite launches MCP1: one URL for every MCP server your team uses — eldadfux · 2026-11-09