EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agent Training

Zishan Xu · hf · 2026-08-07

Training LLM agents for long-horizon tool use typically relies on costly interactions with real or synthesized external environments. To overcome this, researchers introduced EnvACE, an agentic reinforcement learning method that replaces external environment interaction with world rehearsal.

Key mechanisms include:

Role Alternation: The policy first generates a tool call, then plays the role of the environment to produce the induced response, conditioning subsequent decisions on this rehearsed response.
End-to-End Optimization: Both roles are jointly optimized using task-success rewards, allowing the policy to internalize action-environment relationships and form an agent world model for decision-making.

Experiments show that EnvACE achieves strong, transferable performance across benchmarks like BFCL-v4 and tau^2-Bench, outperforming environment-scaling baselines. At test time, the internalized world model enables private rehearsal before execution, yielding further gains without additional external interaction. Code is open-sourced.

Original post →

EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agent Training

More from coding & agent