New study says LLM agent skills create a 59% regression tax across 5,832 runs
TheTuringPost · x · 2026-07-28
Paper finds agent skills can cause a 59% regression tax
A new paper on LLM agents argues that adding more procedural skills is not purely beneficial. Across 5,832 office-task runs, the authors found 553 gains but also 324 regressions: a 59% regression tax on gross gains.
The paper identifies three mechanisms behind these regressions:
- skill-description osmosis: even mentioning a skill can change behavior, sometimes when it is never invoked
- grounding displacement: the skill’s procedure can override how the agent interprets inputs
- verification displacement: the procedure can suppress checks the agent would otherwise perform
The authors conclude that average score alone is misleading. When evaluating skills, teams should track not just what a skill fixes, but which previously solved tasks it breaks and whether the description itself changes behavior.
Related event: Study Reveals 59% Regression Tax in Agent Skills(2 posts)→
More from coding & agent
- Agents are now handling feedback and can PR their own onboarding UI — jasonkneen · 2026-07-28
- A meme about letting the LLM stop inference and call its own Python function — ctjlewis · 2026-07-28
- Meta researcher says coding-agent benchmarks are saturating and need goal-based evaluation — agihouse_org · 2026-07-28
- Gauntlet Loop splits goals across builder agents and a ruthless blind critic — mattshumer_ · 2026-07-28
- Gauntlet Loops are becoming the author’s default workflow for almost every project — mattshumer_ · 2026-07-28
- MCP builders debate when real usage turns into stars, reviews and paid traction — ResponsibleOne6307 · 2026-07-28