AgentSafeEvo: adversarial co-evolution RL hardens LLM agents without hurting capability
Chenxu Wang · hf · 2026-10-15
Researchers from Shanghai AI Lab, Fudan, SJTU and Zhejiang University introduce AgentSafeEvo, a co-evolutionary RL framework for LLM agent safety.
- LLM agents completing multi-turn tool-using tasks can be manipulated by malicious content in queries, tool descriptions, or environment observations
- The framework builds an attacker–defender game: the attacker learns adversarial agent tasks exposing weaknesses in tool use; the defender learns to complete tasks safely under attack
- A learnability-guided task weighting scheme uses within-task defender-reward variability to weight each task's contribution to updates
- Experiments show improved adversarial robustness while maintaining general agent capabilities
Code and data are open-sourced on GitHub and Hugging Face.
More from Safety
- AI Agent Wipes Entire Season of Swedish Reality Show, Premiere Delayed — GaryMarcus · 2026-10-15
- Only 0.51% of Claude conversations touch on elections, Anthropic data shows — _FelixSimon_ · 2026-10-15
- Experiment: Claude models keep shifting decision authority to themselves across 20 rewrite rounds — teortaxesTex · 2026-10-15
- AWS Kiro boxes its AI coding agent in an OS-enforced sandbox on your laptop — shashib · 2026-10-15
- Hacker breached Denmark's national ID database with password "123456", exposing 8.8M people — saibharadwaj · 2026-10-15
- Eichengreen: Concentrated chip supply chains make a frontier AI pause feasible — wfithian · 2026-10-15