Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
Zhaoyang Wang, Qianhui Wu, Haorui Wang, Simon Yu, Baolin Peng, Canwen Xu, Hao Cheng, Wenlin Yao, Boyi Liu, Yite Wang, Jianfeng Gao, Huaxiu Yao
cs.AI, cs.CL, cs.LG
2026-10-20
Open models synthesize 341 GUI+MCP+CLI tasks in WildAWM. GPT-6-Astra scores 59.85% on val. Qwen3.6-35B trained only there goes from 42.84 to 53.42 on eight OOD benchmarks.
Computer-use agents now pick up much of their skill by acting in executable environments. Human demonstrations cost a lot, and they cap the agent at what the demonstrator can already do. A real machine is a poor place to explore: states are hard to reset, live sites change, and a stray click can cause damage.
Hand-built environments can be scored, and they stay small. Automatic synthesis usually locks onto one interface. Browsers, tool calls, and terminals are built as separate projects. WildAWM, from Microsoft Research, UNC, and collaborators, builds one resettable personal computer. Websites, MCP services, and local files share a state. Open-source models do the generating.
A world grows out of a persona. GLM-5.2, running in OpenCode, writes an occupation, named contacts, sites, and a multi-day story, then rewrites any persona that sits too close to one already kept. Day d is planned only after the generator reads the on-disk state from day d-1, so mail and files continue from yesterday.
Activities match a skill library. A skill names the interfaces, database rows, and files that must change. Missing skills and missing MCP services are written on the spot and returned to the library, which ends at 83 skills. A day is kept only if its script replays twice to the same hash. The pages come from 15 WebHarbor mirrors, including Amazon, GitHub, and Coursera, with synthesized records laid on top. The run yields 36 personas, 640 world states, and 8 MCP service types. Median state size is about 52.4K tokens, against about 1.1K for the per-task states CUA-Gym injects. Nearest-neighbor cosine distance is 0.25 versus 0.20.
Every task has to use GUI, MCP, and CLI. GUI clicks pages. MCP, the Model Context Protocol for wrapping services as tools, reads and writes mail and databases. CLI writes and runs scripts. In this paper XAI means that cross-interface loop. It has nothing to do with explainable AI. A programmatic verifier reads the initial state, the final state, and the final answer. Every checklist item has to come out true.
Each task gets six rollouts, half with a hint, from GLM-5.2, DeepSeek-V4-Flash, and MiniMax-M3, on OpenCode, Codex, Claude Code, and three claw harnesses (Nanobot, OpenClaw, Hermes). Five gates decide. Doing nothing must fail. At least one rollout must succeed and must actually use all three interfaces. Among three unhinted rollouts, zero or one success counts as hard, two as medium, and three is sent back to be made harder. The script, a semantic judge, and an agent judge that opens the environment must agree on all six rollouts. The agent judge is then shown the opposite verdict; if it flips, the task is dropped. The probe is aimed at sycophancy. Failures go back to the generator for at most two revisions.
From 2,551 sketches, 341 tasks remain. After the diversity filter, acceptance is about 53%: 273 hard and 68 medium. Unhinted success during synthesis is 21.3%, hinted 45.6%. A task averages 4.2 sub-tasks and 3.1 interface switches, and 74.5% return to an interface already used. Median length is 74 steps. Some GLM-5.2 runs pass 100.
Training uses GRPO, so rollouts of the same task are scored against each other. A fully solved task gets 1. Otherwise the reward adds 0.1 times the checklist pass rate, capped at 1. A format error scores -1, and the gradient hits only that step. If a group has identical rewards and every rollout is a format error, the advantage is -1/sqrt(K), which still teaches format early in training. The split is 100 tasks for a 3-epoch warm start on GLM-5.2 trajectories, 208 for RL (at most 64 steps, 8 tasks per step, 4 rollouts each), and 33 held out for in-domain eval.
Two Qwen models trained only in these worlds improve on all eight out-of-distribution benchmarks. Every model in the table uses OpenCode. Online-Mind2Web, WebArena, and ClawEval are pass rates. The other five are mean scores. The headline average adds those unlike numbers together.
| Model | Avg. of 8 | Comparison |
| Qwen3.5-4B | 30.04 | base 21.83, +8.21 |
| Qwen3.6-35B-A3B | 53.42 | base 42.84, +10.58 |
| GLM-5.2 | 63.21 | teacher for the warm-start traces |
| GPT-6-Astra | 73.11 | strongest model in the table |
The 35B model moves most on ClawEval (38.00 to 54.67, +16.67), OSWorld-Verified (33.16 to 49.22, +16.06), WebArena (53.54 to 67.26, +13.72, tying GLM-5.2), and a 200-task CUA-Gym slice (53.73 to 65.95, +12.22). Online-Mind2Web goes from 69.60 to 80.77, above GPT-5.5 at 72.22 and below Opus-5 at 96.60. QwenClawBench gains only 3.58, and Agents' Last Exam only 2.74. The gap to GLM-5.2 shrinks from about 20 points to about 10. The gap to GPT-6-Astra stays near 20.
The 4B gains sit mostly on Online-Mind2Web (+19.26) and the CUA-Gym slice (+15.96). OSWorld 2.0 moves from 0.87 to 1.03, and ALE from 7.00 to 8.96. Long desktop tasks barely leave the floor.
On the 33 in-domain tasks, GPT-6-Astra's pass@1 is 59.85%, and 27.3% of tasks are still unsolved after eight tries. The trained 35B scores 35.98%, next to the teacher at 37.88% and the base at 5.68%. The 4B model goes from 0.38% to 27.65%. On 100 random rollouts the verifier agrees with the majority of judges 95% of the time, Cohen's kappa 0.90.
Removing any quality gate drops a supervised 4B model by 2.60 to 9.30 points on a four-benchmark average. Dropping judge agreement or the initial-failure check sends the in-domain score to 3.79 or 3.05. Dropping the warm start takes the 35B average from 53.42 to 48.11, a larger hit than removing partial credit (2.25) or the format penalty (1.24).
Switches between interface families rise from 18.4% to 24.7% after supervised training and to 25.5% after RL. Training on the base model's own successes lowers the rate to 15.8%. RL without the warm start lowers it to 15.7%. Inside 40 steps, supervised training and the full recipe solve similar shares, 34.9% and 36.8%. From 40 to 120 steps the supervised model adds 9.7 points and the full recipe adds 17.0, reaching 53.7% solved by 120 steps.
Environments are the scarce input for agent RL. The other synthesis systems in the comparison mostly call GPT-5, Opus, Gemini, or Sonnet, and few of them expose clicking, tools, and the shell in one world. WildAWM turns out 341 long tasks with programmatic rewards, using open models only.
All eight benchmarks move, so the gain is not confined to the synthetic split. At 35B, WebArena ties the teacher, GLM-5.2. The optimizer is stock GRPO. Cross-interface behavior is copied from 100 teacher trajectories. RL is what extends completion past 40 steps. The 4B model still scores near zero on OSWorld 2.0. More worlds do not replace capacity.
OOD tasks split into quartiles by how often frontier models solve them. Gains sit on the middle two, at +19.9 and +12.2. The hardest quartile does not clearly move. On the in-domain set the trained 35B model stays next to GLM-5.2, 35.98% versus 37.88%, well short of GPT-6-Astra at 59.85%. The abstract's figure of about 60% matches this 33-task validation set. Per-task rates on all 341 are not listed.
The eight-benchmark average mixes pass rates with graded scores. WebArena uses 226 of 812 tasks, CUA-Gym 200 of 10,910, OSWorld-Verified 99 of 330, and Agents' Last Exam a 99-task Linux subset. Policies that cannot see images get a text description of the screenshot from Qwen3.6-35B-A3B, so the 4B GUI score borrows a larger model's vision. The benchmark LLM judge is GPT-5.5. The open-model claim covers synthesis, not that judge. The open models also run under NVFP4 quantization.
The 95% agreement is measured after a gate that already requires the verifier to match the judges. The counterfactual probe only catches an agent judge that changes its mind. If all three judges are wrong together, the task still passes. Kappa between GPT-5.6-Sol and Opus-5 is 0.79. The gate ablation covers only supervised training of the 4B model. Removing the solvability gate raises Online-Mind2Web to 44.87, above the full pipeline's 38.49. Removing the probe raises OSWorld-Verified to 24.09, above 17.17.
The first six personas, used to debug the pipeline, account for 178 of 341 tasks. One accepted task costs about 69.8 million generator input tokens on average. The web pages are existing mirrors. Nobody rated whether these machines feel like computers a person has used for many days, and nothing shows that the trained agents cause fewer accidents on real systems. The security worry in the introduction is not measured again.