Berkeley paper: self-rewriting agents gain benchmark points while silently losing solved tasks

rohanpaul_ai · x · 2026-11-09

A UC Berkeley and collaborators paper examines whether agent harness evolution—recursive self-improvement with frozen weights by editing prompts, skills, tools, and code—yields reliable gains.

Key findings:

All 16 evolved agents gained new task coverage while losing previously solved tasks; average scores hid the regressions. One general-task case gained 5.2 reliability points while dropping ten previously covered tasks.
Most reliability gains also raise deployment cost.
Specific changes beat generic reminders, but combining them can flip benefits; the highest-scoring agent never delivered the largest mean improvement, and stricter admission discarded useful alternatives.
Transfer is selective: coverage usually falls on general-task targets but rises on coding targets with outer-layer-only harnesses.

The authors propose preservation-aware evaluation, tested decision rules, and explicit applicability conditions, plus EvoAudit, which turns diagnostics into concrete revisions.

Original post →

Berkeley paper: self-rewriting agents gain benchmark points while silently losing solved tasks

More from coding & agent