Black-box attacks steal agent skills with 48% exact recovery, study finds
rohanpaul_ai · x · 2026-08-17
A new study tests whether proprietary SKILL.md can be stolen via black-box interaction with agents. Across 5 commercial models, even plain extraction prompts averaged 48% exact recovery and a 0.91 LLM-judged leakage ratio. Chain-of-thought prompts pushed exact recovery to 72%. Blocking verbatim copying is insufficient; translation and rewriting attacks often drove exact match to 0% while preserving meaning. Strongest defenses stop exact disclosure but semantic leakage persists. Platforms should treat skill contents as exfiltratable data.
More from Safety
- New brief: companies may break the law when AI chatbots mislead customers — PolarBearby · 2026-10-16
- US HHS floods the zone with clinical trial reform, from Operation TrialBlazer to ARPA-H — anshulkundaje · 2026-10-16
- 66% of billion-dollar companies leave critical bugs unpatched, deflating AI bug-hunting hype — rseroter · 2026-10-16
- Musk says SpaceX will likely follow Anthropic in opening up to third-party safety evaluators like METR — elonmusk · 2026-10-16
- Controversial take: If EA doomers truly believe AI risk, why not stage a controlled AI hacking incident — AaronBergman18 · 2026-10-16
- CMU's Andrea Bajcsy on what safety means for generalist robots, beyond collision avoidance — CSProfKGD · 2026-10-16