EvoHarnessBench shows adding tools to agents can cause forgetting of solved tasks
mohitban47 · x · 2026-09-12
Researchers introduce EvoHarnessBench, a benchmark for self-evolving agents facing a neglected form of continual adaptation: the harness itself evolves as tools, skills, and specialist agents are progressively added, while previously solved tasks stay in the evaluation.
Key findings:
- Harness expansion induces forgetting — each capability upgrade can degrade performance on tasks the agent already solved;
- Current self-evolving methods fail to recover reliably: today's agents cannot consistently retain earlier competence while adapting to new capabilities.
The benchmark probes the trade-off between growing capability and retaining old skills, suggesting that stacking features onto agentic systems is not a free upgrade.
Related event: EvoHarnessBench: Adding More Tools to Agents Can Make Them Worse(2 posts)→
More from coding & agent
- One-person companies are about the org chart, not headcount: humans judge, agents execute — ciguleva · 2026-10-28
- OATs: open-source toolkit turns everyday objects into personality-bearing agents — Stefania_druga · 2026-10-28
- Dev: local coding dropped from 100% to 5% in a year as cloud agents take over — bytebot · 2026-10-28
- Open-sourced 501(c)(FREE): agent-built free back office for US nonprofits — gralston · 2026-10-28
- Grok Bot 1.0.0 early build reveals Shared Bots, service accounts and team workflows — XFreeze · 2026-10-28
- Anne Griffin hosts 20-person SF salon on the product decisions behind AI agents — annetgriffin · 2026-10-28