SkillSonar cuts agent attack success from 48.2% to 10.4% with runtime guard skill

rohanpaul_ai · x · 2026-09-24

The paper "Defense-as-Skill" argues malicious agent skills can look safe at install time and only turn dangerous mid-task once the agent gains access to files, tools, credentials, or external services — so pre-install scanning alone is insufficient.
It proposes SkillSonar, a safety skill that checks sensitive actions at runtime and decides whether to allow, narrow, replan, or ask the user first.
On GLM-5 it cut attack success from 48.2% to 10.4% on known attack types and from 60.6% to 11.5% on unseen risk families.
A key finding: merely installing the safety skill was far weaker — the agent had to be explicitly told to consult it before acting.
Recommended stack: scan skills pre-install, check actions during execution, and keep hard protections like permissions and sandboxing underneath.

More from coding & agent