OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution
Yunhao Chen, Xin Wang, Yixu Wang, Yi Liu, Jie Li, Yan Teng, Xingjun Ma, Xia Hu, Yu-Gang Jiang
cs.CL
2026-08-01
OpenART red-teams the shared environment state, not the prompt, hitting 85.0% attack success across 75 agent-model configs and beating instruction-only attacks by up to 17 points on hard tasks.
Most agent safety benchmarks still use the old recipe of short tasks behind a single interface: hand the model one tool call and see whether it refuses a malicious instruction. ToolEmu and its peers average one to four tool calls. Real agents run long-horizon workflows, repeatedly reading and writing a shared environment (workspace, memory, plan). A state change planted early propagates through dozens of calls and surfaces as unsafe behavior much later. These benchmarks also each ship their own interface, so you cannot directly compare Claude Code with Codex. OpenART (Fudan, Shanghai AI Lab, XSafeAI) fills the gap by making the red-teaming target the executable environment rather than the prompt, and by building an arena that evaluates 15 real agents across 5 foundation models under one protocol.
OpenART has three stages. Scenario construction synthesizes over 10,000 stateful scenarios across 50 domains from a pool of more than 500,000 tools, MCPs, and skills; each carries a benign task objective, an executable environment, and a hidden safety contract, with a median of 97 tool calls and dependency depth 32. Cross-agent projection keeps scenarios target-agnostic until runtime, when adapters wire them into each agent's native interface, so one scenario runs across 15 agents (OpenCode, Aider, Claude Code, Codex, through Qwen Code) paired with 5 models (GPT-5.5, Opus-4.8, GLM-5.2, Qwen-3.7-Max, DeepSeek-V4-Pro), 75 configs in total. The attack surface has eight vectors: workspace, instructions, skills, tools, MCPs, short-term memory, plan state, long-term memory.
The core idea is environment evolution: the task objective and safety contract stay fixed, and only the target-visible environment state changes. The attacker EMHA is a black-box policy that updates no model parameters, with both attacker and target frozen. It models coordinated state changes as a hypergraph where vertices are attack subgoals and hyperedges link prerequisites to successors; evaluator feedback is redistributed along paths and soft Q-learning updates path values, while an archive retains the highest-fitness attacks and evolves offspring. This works because an injected state change is first consumed at about 23% of execution, yet the first unsafe output appears at 64%, with a median propagation of 37 actions. Instruction-only attacks cannot reach this latent risk.
Pooled Strict ASR (attack success rate) reaches 85.0% across all 75 configs. The spread by foundation model is large: Qwen-3.7-Max and DeepSeek-V4-Pro are easiest (94.6%, 94.7%), GPT-5.5 and GLM-5.2 sit in the middle (88.5%, 87.9%), and Opus-4.8 is the hardest (59.2%).
| Model | ASR |
| Qwen-3.7-Max | 94.6% |
| DeepSeek-V4-Pro | 94.7% |
| GPT-5.5 | 88.5% |
| GLM-5.2 | 87.9% |
| Claude-Opus-4.8 | 59.2% |
| All-config avg | 85.0% |
Environment evolution's edge over instruction-only attacks scales with complexity: 1.8 to 2.7 points on the simplest scenarios, 17.2 to 17.6 points on the most complex. Over five evolution rounds on DeepSeek-V4-Pro, ASR climbs from 42.9% in round one to 94.7%, a cumulative 51.8 points. A variance decomposition attributes 73.6% of ASR variation to model plus capability, and an additional 7.6% to agent-runtime identity (Claude Code vs Aider around the same model).
Two takeaways for practitioners. First, when you deploy agents the attack surface is not only prompt injection; it is the state accumulating in workspace, memory, and plan. A safety eval that only checks whether the model refuses a bad instruction misses this long-horizon drift. Second, the runtime you wrap around a model changes its safety profile, not just its capability. "We run Opus so we are safe" does not hold; the framework itself accounts for roughly 7.6%. The 85% figure is more a measuring stick than a verdict: against an adaptive adversary that can touch the environment, current agent stacks fail most of the time, and Opus at 59% is the outlier worth studying.
The authors concede that although agent implementation clearly affects safety, they do not identify the mechanism, for example why Claude Code differs from Aider on the same model. Scenario admission requires a deterministic evaluator plus a GLM-5.2 judge, which can introduce selection bias. The paper does not examine how environment evolution interacts with different alignment or safety-training methods. ASR depends heavily on attack budget and evolution rounds; five rounds are needed to climb from 43% to 95%, and a defender could argue the picture differs at lower budgets, though the monotonic climb is itself the warning. The safety contract is the framework's own, and whether it matches a real deployment's policy is unaddressed.