Microsoft study: Agent skills can be harmful, 307 failures reveal pitfalls

coallaoh · x · 2026-08-16

Microsoft Research and collaborators published a paper titled "Agent Skills Can Be Harmful" that systematically investigates how agent skills can cause task failures and efficiency regressions.

Found 307 skill-induced failures on SkillsBench and SWE-Skills-Bench: 125 functional failures and 182 efficiency regressions.
Surprisingly, most failures are not caused by obviously irrelevant skills; seemingly relevant skills often make the agent over-rely on templates and default practices, missing task requirements. 68.8% of functional failures fall into this category.
Efficiency issues mainly stem from skills causing agents to add unnecessary steps, such as repeated validation and heavy implementation procedures.
The authors propose SkillTriage, a tool for attributing and categorizing skill-induced problems.
The study reminds that the value of skills should not be judged solely by whether they are invoked, but also by their actual help in specific tasks; future systems need evaluation, regression testing, and retirement mechanisms.

Original post →

Microsoft study: Agent skills can be harmful, 307 failures reveal pitfalls

More from coding & agent