Amazon paper: stale agent skills cut pass rate 11.4 points, selective maintenance fixes it

rohanpaul_ai · x · 2026-11-20

An Amazon/USC paper shows saved agent skills can silently go stale as environments change: on Terminal-Bench 2, a skill that had added 4.7 points for Claude Opus 4.7 left it 11.4 points below the no-skill baseline after a change; Haiku 4.5's IRS tax-form field accuracy dropped to 30.5% with a stale skill.

Their fix, selective maintenance, replays saved checks without extra model calls, repairs only conflicting skill content, and rebuilds when local repair is insufficient — lifting accuracy by up to 67pp over reusing stale state and cutting execution steps by up to 37%. Post-maintenance, Haiku's tax-form accuracy reached 97.5%.

Practical takeaway: validate saved agent skills against the current environment before reuse, instead of waiting for tasks to fail.

Related event: Amazon Paper Shows Stale Agent Skills Can Drop Performance by Over 11 Points(2 posts)→

Original post →

More from coding & agent

coding & agent channel →