From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation
Quanyu Long, Xiao Chen, Jianda Chen, Haozhen Zhang, Qisheng Hu, Jianzhu Bao, Wenya Wang
cs.CL
2026-10-05
Trace2Env turns traces into a worldbook a world-model agent runs as the environment, lifting GPT's nine-env average from 69.51 to 75.94 and ALFWorld replay success from 3% to 85%.
Agents need an environment that answers an action and keeps the consequence. Terminals, software workspaces, and internal systems are often impossible to stand up again once the code or machines are gone. Interaction logs usually remain.
A language world model (LWM) predicts the next observation from a description, the history, and the current action. Latent state breaks that shortcut. Deleting report.txt can print nothing, while a later read has to fail. The simulator has to record the deletion when it happens. A flat prompt of old observations does not.
Terminal-Universe rebuilds an executable terminal from trajectories. Complex systems still depend on implementation details the logs never had. Trace2Env keeps a behavioral specification, and a world-model agent plays the environment.
Offline, gpt-5.6-sol reads construction traces once and freezes a worldbook. Terminal uses 20 Terminal-Bench trajectories. Web uses 50 WebArena trajectories. Android and SWE have no runnable original system, so each test row uses a worldbook built from the other folds of a task-grouped five-fold split.
The book has four layers. Schemas name actions and state fields. Grounded evidence keeps aligned transitions, a few demonstrations, and the original observation text. Induced abstractions store recurring effects, constraints, and observation contracts. Provenance points each artifact at the supporting turns. Effects with ambiguous attribution are dropped. A rule is executable only after review against success-failure contrasts and counterexamples.
At run time three resources stay separate. The worldbook is read-only and shared, and it says how the environment behaves. Episode state says what is true now. A missing field is treated as unknown. Episodic memory keeps the action-observation text. A file seen in a construction trace does not exist in a new episode for that reason.
The world-model agent retrieves book entries, labels them supporting, uncertain, or format-only, and proposes ordered state effects plus an observation. It does not edit state itself. A shared harness checks the proposal on a copy against schemas, rules, and constraints, and commits only if the checks pass. On rejection, the reference policy drops the effects and can still keep the observation. Model weights stay frozen.
Single-step tests cover Terminal, SWE, Android, and Web from AgentWorldBench, plus Food, Shopping, and Benefits from EnvScaler. A gpt-5.2 judge scores format, factuality, consistency, realism, and quality from 1 to 5. The official protocol rescales those scores to 0-100 and averages them. Multi-turn tests use 100 ALFWorld tasks and 40 SciWorld tasks. Actions taken in the world model are replayed in the real environment. That success rate is W2R. CR is W2R divided by real-environment success.
Baselines are direct prompting, lexical retrieval of five raw turns, the worldbook pasted into the prompt, and the runtime harness with the book removed. Every book is built once with gpt-5.6-sol and reused for both backbones.
| Method | GPT avg | DeepSeek avg | ALFWorld replay | SciWorld replay |
| Direct prompting | 69.51 | 68.71 | 3% (CR 0.032) | 45% (CR 0.529) |
| Raw-trace retrieval | 73.76 | 73.06 | not reported | not reported |
| Static worldbook prompt | 74.35 | 74.86 | not reported | not reported |
| Harness only | 70.79 | 70.35 | not reported | not reported |
| Trace2Env | 75.94 | 75.75 | 85% (CR 0.914) | 60% (CR 0.706) |
Against direct prompting the average rises by 6.43 on GPT and 7.04 on DeepSeek. Trace2Env is best in 11 of 14 environment-backbone cells and is never below direct prompting. Factuality, consistency, realism, and quality rise in all 14 cells. Format drops only for Android with GPT and Web with DeepSeek. On the four AgentWorldBench environments, trajectory-clustered intervals for the gain exclude zero.
Absolute scores are uneven. On GPT, Terminal moves from 55.76 to 65.42, and factuality from 41.7 to 50.5. Web moves from 54.23 to 59.55, with factuality still at 43.6. Benefits moves from 88.89 to 98.11. The full system is not first everywhere. On Shopping with GPT, raw retrieval scores 86.07 and Trace2Env 85.36. On Terminal with DeepSeek, a static worldbook prompt scores 65.24 and Trace2Env 64.12.
In-simulator success and replay success split. Direct prompting reaches 97% and 92.5% inside the two games, then replays at 3% and 45%. Trace2Env scores 88% and 87.5% inside, and 85% and 60% on replay. Real success is 93% and 85% for both. An unsupported put is a no-op in real ALFWorld. Direct prompting narrates a successful placement, the task agent continues from that false inventory, and replay fails. Trace2Env keeps the no-op and the original inventory, the help text exposes a legal move, and replay finishes.
On the Terminal ablation, 354 rows, the full system scores 64.89 in that table and 65.42 in the main Terminal cell. Schema only scores 62.05. Adding abstractions lowers the score to 60.69. Adding evidence raises it to 64.18, a paired gain of +2.45±0.89. Abstraction on top of evidence adds +0.69±0.63. Evidence on top of abstraction adds +4.26±1.02. Removing adaptive inspection costs 1.79±0.91. Removing state tracking costs 0.97±0.78.
The trace curve is still rising: 64.89 at 20 traces, 66.42 at 36. Main experiments use 20. Full Trace2Env on Terminal spends about 5 calls and a list price of $0.187 per row. Dropping the adaptive loop keeps about 80% of the gain over direct prompting at about 29% of the inference cost.
Gains remain on tasks absent from construction, and the size depends on the backbone. On 312 Terminal rows, versus harness only, GPT is +3.69 (interval 1.71 to 5.63) and DeepSeek is +0.61 with an interval through zero. On 151 Web rows versus that same baseline, GPT is +4.21 and DeepSeek is +5.00, both intervals clear of zero. On 378 SWE rows versus direct prompting, GPT is +5.09 and DeepSeek is +6.60, again clear of zero.
If the original system is gone and the logs remain, a behavioral spec is easier to stand up than a reverse-engineered sandbox. The book and the runtime model can differ. A gpt-5.6-sol book run by deepseek-v4.1-flash averages 75.75, close to 75.94 from the model that built the book. For evaluation, replay is the number to read. In-simulator success favors a prompted model that can maintain a false state.
The single-step gain is modest: 1.59 over a static worldbook prompt and 2.18 over five-trace retrieval, on the GPT average. Multi-turn state, and simulation that no longer asks the original environment for feedback, is the split from retrieval-style LWMs. WorldEvolver still needs later real outcomes to edit its memories and rules. Using this kind of simulation to train a task agent is untested.
Fidelity stops at behavior the traces showed. The conclusion says so, and says some backbones use the reconstructed evidence less consistently than others. On unseen Terminal tasks, DeepSeek's gain over the harness includes zero in its interval. Shopping is built from 3 successful trajectories and 55 turns. Food, Benefits, and both text games are also skewed toward successful or gold-path traces. On Android, when an app appears in no other fold, GPT sits 0.59 below direct prompting across 127 rows, and no interval is given.
Multi-turn consistency is measured only on ALFWorld and SciWorld. Terminal, Web, SWE, and Android stay at next-observation prediction. SciWorld replay is 60% against 85% real success, so CR is 0.706. A rule shortcut that needed a template and confidence of at least 0.9 never fired in the reported books. After a rejected proposal, the reference policy can still write the proposed observation into memory. Scores come from a gpt-5.2 judge, with no exact match to the real observation and no head-to-head against a trained LWM.