Study finds Agent skill injection may lower Pass@2 rates
rohanpaul_ai · x · 2026-08-27
Research indicates that attaching a skill file to every prompt in a coding session is often a net loss. The WebDev-Skills-Bench shows that while skill injection raises token costs by at least 72%, it lowers the mean Pass@2 by 1.3 to 4.2 points across four models. Losses concentrate on easy early tasks, likely because skills lock in structural choices, turning recoverable mistakes into chain-terminating failures.
More from coding & agent
- 18 Local Apps Behind One MCP Endpoint: Tool-Search Architecture — jarjav69 · 2026-08-27
- Comparing Agent Frameworks: Governance in CircleChat, Buzz, Duet — Agreeable_Craft_8943 · 2026-08-27
- AI and Git: Why Granular Commits Matter More Than Ever — Gleb_SV · 2026-08-27
- Preventing AI Agents from Hallucinating Function Arguments — Jay299792458 · 2026-08-27
- MCP server lets AI agents consult Tarot, I Ching, runes, and more — Puzzled_Most_5365 · 2026-08-27
- Solo coding 1.68M lines in 414 days: A journey with AI agents — jamestagg · 2026-08-27