New study says LLM agent skills create a 59% regression tax across 5,832 runs
TheTuringPost · x · 2026-07-28
Paper finds agent skills can cause a 59% regression tax
A new paper on LLM agents argues that adding more procedural skills is not purely beneficial. Across 5,832 office-task runs, the authors found 553 gains but also 324 regressions: a 59% regression tax on gross gains.
The paper identifies three mechanisms behind these regressions:
- skill-description osmosis: even mentioning a skill can change behavior, sometimes when it is never invoked
- grounding displacement: the skill’s procedure can override how the agent interprets inputs
- verification displacement: the procedure can suppress checks the agent would otherwise perform
The authors conclude that average score alone is misleading. When evaluating skills, teams should track not just what a skill fixes, but which previously solved tasks it breaks and whether the description itself changes behavior.
Related event: Study Reveals 59% Regression Tax in Agent Skills(2 posts)→
More from coding & agent
- theo builds his own visualizer for today's agent models, showing how cheap Luna really is — ivan_bezdomny · 2026-09-23
- Vite+ Hits RC: One Rust-Powered CLI to Replace Your Entire Web Toolchain — cnakazawa · 2026-09-23
- Tesla's in-car Grok agent books trips across Gmail, Calendar and Notion in one command — xiaohu · 2026-09-23
- Tesla's In-Car Grok Assistant Now Executes Cross-App Tasks in One Sentence — xiaohu · 2026-09-23
- Garry Tan says Capy lets him ship PRs much faster than Codex or Claude Code — garrytan · 2026-09-23
- DeskPilot: open-source native Python desktop client for local LLMs with MCP and sandboxed tools — poofph · 2026-09-23