EvoWorld: Towards Out-of-Sight World Evolution in Generative Video Models
Yequan Bie, Dengyang Jiang, Zhixuan Chen, Xiaozhuan Liang, Haoxuan Che, Quande Liu, Xi Chen, Hao Chen
cs.CV, cs.AI
2026-11-05
A VLM world brain tracks world state, reasons about off-screen events, and steers a video renderer; core task success hits 90.8% on the new benchmark vs 21.7% for the best baseline.
Video world models have gotten much better at long-horizon consistency: recurrent state memory, retrieval-based memory, geometry-grounded 3D memory. Every one of these mechanisms solves "don't change what was already seen." None of them handles "the world keeps evolving where the camera isn't looking."
The opening example makes it concrete. A camera frames a vendor on a pedestrian overpass, pans away while a glass facade collapses nearby, then pans back. The plausible outcome is the vendor crouching beside his cart. Existing models give you a vendor who stands still, follows the camera, or gets duplicated. HKUST, Tencent and CUHK formalize this as a new task, Out-of-Sight Causal Evolution: an event is injected mid-rollout after the subject leaves the frame, and the revisit must reveal a causally plausible reaction. Visual consistency with the last observed frame does not count.
The task abstracts a rollout into three parts: initial (frame the subject), event (camera pans away, the prompt switches), revisit (camera returns and reveals the subject's new state). EvoWorld splits reasoning from rendering:
The data is synthetic by design. Gemini3.1-Pro expands structured scenario tuples (subject, scene, event, compatible reaction, camera) into single-take scripts; Seedance 2.0/2.5 renders them; a VLM verifies shot continuity, subject identity, event visibility and reaction visibility, while Lucas-Kanade tracks with a RANSAC affine fit reject clips with broken camera motion. The result is 12k videos of at least 10 seconds covering 19 event families (65 fine-grained types), 63 scene types and 14 reaction types across photorealistic and game domains. EvoWorld-Bench has 120 cases, screened against the training set with both frame-level visual retrieval and prompt-level semantic similarity.
Main table (EvoWorld-Bench, 120 cases):
| Method | Core | Revisit | Event | Causal | React. |
| Evoke-Turbo (2nd best) | 21.7 | 98.3 | 89.2 | 32.5 | 29.6 |
| EvoWorld | 90.8 | 100.0 | 97.5 | 94.2 | 85.0 |
The other five baselines (LingBot-World v1/v2, YUME-1.5, SANA-WM, Echo-WM, Evoke) all land between 13.3 and 20.0 on Core, which requires revisit, event realization and full causal plausibility in the same rollout. The Evoke-Turbo row is the interesting one: 98.3 Revisit and 89.2 Event, yet 21.7 Core. Camera control plus event rendering does not produce a causal chain.
Ablations: IAA lifts Core from 82.5 to 84.2; stage-2 online rollout training takes it to 90.8 and Causal Plausibility to 94.2. The contribution split is clean. Feeding the brain's reasoning prompt to Evoke-Turbo doubles its Core to 45.0; conversely, feeding the trained renderer a blind revisit prompt (no reasoning) drops performance sharply while keeping it above the baselines. Reasoning and renderer training each carry about half the gain, and the prompt transfers across renderers. One-shot references: LTX-2.5 given the entire story as a single composed prompt reaches 34.2 Core, MiniMax-H3 only 21.7 (100.0 Revisit, 99.2 Event, but 44.2 Causal). Segment-autoregressive generation with reasoning is the right shape for this task.
The task definition is itself a contribution: it turns "does the model have state and causality" into a measurable benchmark, directly usable for games, simulation and agent environment generation. The architectural read is worth stealing: a pure diffusion renderer should not own world state; a VLM brain plus a specialized renderer is a cheaper division of labor, and the transfer result shows the approach is not tied to one renderer. Data, benchmark and code are released, so the entry bar for follow-up work is low.
The authors' own list (Appendix G): single event followed by a single reaction only, no interacting events or multi-entity evolution; camera motion limited to horizontal pan-away and pan-back, with large displacement, orbiting and 6DoF trajectories untested. A few more caveats the paper does not discuss. The main table pits a full system against bare models: baselines execute the blind revisit prompt directly while EvoWorld adds VLM reasoning and entity retrieval (Table 3 partly answers this, but the renderer was also trained on their 12k in-domain videos, which no baseline saw). The judge is Qwen3.8, the same family as the brain, and self-preference bias is never mentioned. Training data is entirely Seedance-generated, so its distributional quirks propagate into the renderer. And the benchmark is 120 cases.