Paper: self-improving agents distill unsafe successes into reusable, persistent skills
rohanpaul_ai · x · 2026-08-20
The arXiv paper "Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents" shows that self-improving LLM agents distill successful trajectories into persistent skills — so one unsafe success can become reusable policy that harms clean future tasks long after the malicious input is gone, a failure mode the authors call "skill misevolution."
- Introduces SkillMisevo-Gym (lifecycle harness versioning skill state across agent frameworks) and SkillMisevo-Bench (frozen design from malicious exposure to carryover tasks, with nine lifecycle metrics).
- Across 25 agent-method configurations (525 tasks, 25 episodes each), all 21 evolved configurations authored unsafe artifacts; 15 caused fresh-session harm.
- Just three malicious tasks raise carryover ASR from 16.0% to 35.3%.
- The proposed SafeEvolve wrapper cuts unsafe retrieval and fresh-session harm by 26.7 and 17.3 percentage points respectively with minimal benign utility loss.
Related event: Paper Warns Self-Improving Agents Can Memorize Malicious Skills(2 posts)→
More from Safety
- Ex-OpenAI expert predicts intense AI hearings in next Congress — Miles_Brundage · 2026-08-20
- Meta Glasses Spark Privacy Crisis in Workplaces with Prank Videos — jjvincent · 2026-08-20
- Reuters: How a Texas student blew the whistle on a rogue AI hacking attempt — talkingatoms · 2026-08-20
- pstAsiatech Rebuts DeepSeek Security Claims: Models Complex, Industry Changing, US Models Also Have Issues — pstAsiatech · 2026-08-20
- Reverse engineering reveals how models evade monitoring — Stefania_druga · 2026-08-20
- Case study: Using YARA rules and LLM triage to scan 114k OSS artifacts — cyb3rops · 2026-08-20