Shanghai AI Lab's SHE and SafeEvolve cut LLM agent attack success rate from 17.1% to 5.5%

机器之心 · wechat · 2026-09-17

As LLM agents enter browsers, email, and business systems, security checks that only inspect final replies miss risks at any execution step—hidden web instructions, over-broad tool permissions, and corrupted memory. Shanghai AI Lab with Fudan, SJTU, HKUST, and Zhejiang University propose SHE and SafeEvolve.

SHE: trajectory-driven Harness evolution

Decomposes the safety Harness into SystemPrompt, RuleBank, SafetyMemory, and ToolPolicy for targeted, verifiable, rollback-able updates
Diagnoses full execution traces to pinpoint where constraints failed and which component to revise
On Agent-SafetyBench: average attack success rate drops from 17.1% to 5.5%, task utility under attack rises from 31.6% to 47.6%; on unseen AgentHarm, harm score falls from 19.8% to 9.8%, with boundaries transferring to other agent models

SafeEvolve: absorbing safety experience into Policy

Extracts a SafetyPrompt and layered SkillBank on the Harness side, retrieving relevant skills at inference
Trains Policy via Harness-use SFT then Harness-augmented RL, with a Verifier scoring task completion, dangerous behavior, and tool calls
On Qwen3.5-4B, AgentDojo attack success drops from 2.37% to 0.79% while clean utility improves; AgentHarm harm score falls from 56.45 to 12.27

Together they advance agent safety from one-time static configuration toward continuous, trace-diagnosed co-evolution of Harness and Policy. Papers: SHE, SafeEvolve

Original post →

Shanghai AI Lab's SHE and SafeEvolve cut LLM agent attack success rate from 17.1% to 5.5%

More from coding & agent