Self-improving agents should only accept harness edits that beat the current version on fresh tasks
rohanpaul_ai · x · 2026-10-16
A new paper studies self-improving agents that rewrite their own harness — prompts, memory, tools, and surrounding code — where accepted edits persist across runs, so a bad edit can overfit, erase old skills, or spread to other users. The authors map harness self-improvement onto evolutionary algorithms, from simple keep-the-best search to archives, Pareto selection, and bandit-guided budgets. Their admission rule: keep an edit only if it beats the current harness on fresh, independent tasks by a set margin. In a compiled case, GPT-5.4-nano with an evolved harness and skills scored 77.58% on SpreadsheetBench vs. 76.20% for GPT-5.5 with skill tuning alone (costs not matched). They advise keeping evaluators, permissions, and emergency stops out of the agent's reach.
Related event: New paper formalizes rules for self-evolving agent harnesses(2 posts)→
More from coding & agent
- Perplexity open-sources SPACE, the production sandbox platform behind Perplexity Computer — AravSrinivas · 2026-10-16
- Meta releases Muse Spark Open: 816B MoE, 42B active, 1M context, day-0 on Baseten — baseten · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16
- Vercel makes Python generally available: FastAPI, Django, Fluid Compute support — steren · 2026-10-16
- Full course: build a playable Unity fishing game with Claude Opus 5.5 in six steps — Roger_M_Taylor · 2026-10-16
- Marc Lou praises PostShiba, a cute indie alternative to AWS SES for email infra — marclou · 2026-10-16