arXiv paper: attributing 307 LLM agent failures to specific skills via differential analysis
alex_verem · x · 2026-08-21
The arXiv paper "Agent Skills Can Be Harmful" (2608.11888) systematically attributes agent failures to loaded skills. The authors introduce a differential analysis framework that attributes a failure or cost regression to a skill by comparing a skill-guided run against a no-skill or semantically matched reference run, instantiated on SkillsBench and SWE-Skills-Bench, yielding 307 skill-induced failures (125 functional failures, 182 efficiency regressions).
The paper also builds SkillTriage, a taxonomy-guided attribution tool that normalizes paired cases, extracts differential evidence, and produces triage reports. Major findings: functional failures are rarely caused by obviously irrelevant skills — seemingly relevant skills often make agents incorrectly implement or omit required elements, while efficiency regressions show up as increased token use and execution time.
Related event: Study Finds AI Agent Skills Can Cause 307 Task Failures(2 posts)→
More from coding & agent
- GitHub Launches Agent Plugins 1.0 for Cross-Client Portability — JeremyCMorgan · 2026-08-22
- Panasonic Avionics Accelerates Aircraft Diagnostics with Agentic AI on AWS — AWS ML Blog · 2026-08-22
- MiniMax-H3 video inpainting ported to diffusers modular blocks: 6-step subject swap — linoy_tsaban · 2026-08-22
- LangChain Academy Tutors: Learn with coding agents — LangChain · 2026-08-22
- AI Social Media Agent Shows 10x Higher Engagement Than Human — RichardsonDx · 2026-08-22
- Fable Workflow: Orchestrate via High, Delegate to Subagents — dotey · 2026-08-22