Amazon paper: stale agent skills can drop performance 10pp and raise cost 40%
rohanpaul_ai · x · 2026-11-20
Amazon's new paper When Agent Skills Go Stale studies how agent skills behave after environment changes. Skills store past experience (file paths, field names, commands) to amortize inference cost, but silently go stale when apps or APIs change.
Findings across 4 task settings and 2 models (Claude Opus 4.7 and Haiku 4.5):
- Previously beneficial skills can reduce task performance by over 10pp or increase execution cost by nearly 40% vs. no skill; success rates can hold while execution savings vanish.
- Impact depends on which environment-specific details became outdated and the agent's ability to compensate using current observations.
The authors propose selective maintenance: replaying checks from successful executions without extra model calls — verifying whether observed differences conflict with skill state, repairing affected content, validating updates, and rebuilding when local repair fails. vs. reusing stale state, maintenance improves accuracy by up to 67pp and cuts execution steps by 37%, with no description of the change supplied.
More from coding & agent
- Beyond Model Releases: Effect, Alchemy and Paper Reshaped This Dev's Workflow — samgoodwin89 · 2026-11-20
- Google Antigravity shipped 150 releases in 90 days — and it's finally competitive — rseroter · 2026-11-20
- Gemini Turns One Static Image Into a 60fps Interactive 3D Scene, On Its Own — mtizard · 2026-11-20
- 200 lines of Python + Gemini generates a synced-caption, voice-cloned walkthrough video — DynamicWebPaige · 2026-11-20
- Strategist evolves reasoning strategies as code, wins 24/30 benchmarks at 39% lower cost — liuzhuang1234 · 2026-11-20
- RestTemplate is deprecated in Spring Boot 4.2 — here's the direct swap to RestClient — therealdanvega · 2026-11-20