SkillHarm benchmark shows agent skill-poisoning attacks succeed up to 86.3% of the time
hhsun1 · x · 2026-10-27
SkillHarm, presented at EMNLP 2026's LLM Agents session, introduces a benchmark for skill-based attacks across the agent skill-use lifecycle. It defines 12 risk types spanning data pipelines, system environments, and agent autonomy, with two attack scenarios: Fixed-Payload Poisoning and Self-Mutating Poisoning (where benign executions silently mutate persistent skill content). The authors built an automated pipeline using coding agents with natural-language harnesses, producing 879 attack samples across 71 skills. Current agents prove highly vulnerable: up to 86.3% attack success for FPP and 69.3% for SMP. Notably, many apparent failures stem from agents not engaging with the poisoned file rather than genuine resistance.
More from coding & agent
- Batch-editing hundreds of drone videos by scripting DaVinci Resolve with Astra — sethlazar · 2026-10-27
- DeepSeek V4.1 Flash lands in Cursor, dominating the cost frontier at $0.60-$0.98 per task — dean_rie · 2026-10-27
- Masking slides for AI: placeholders that survive the model but break the layout — shoumikgoswami · 2026-10-27
- Creator has Claude edit her iPhone Duo review inside Tella's agent video editor — evielync · 2026-10-27
- Ant International and Westlake University release Trajectory2Gym for tool-use agent training — ant-intl · 2026-10-27
- Eight universities propose unified framework for work agents — Zishan Xu · 2026-10-27