WorldDraft: If Code Builds the World, Can Video Models Just Render It?
Zhiyu Hou, Wen-Jie Shu, Harold Haodong Chen, Weilin Ruan, Chenfei Liao, Liuzhou Zhang, Disen Lan, Xiaogang Xu, Qi Chen, Ser-Nam Lim
cs.CV
2026-11-12
WorldDraft tests 12 video models on 432 paired cases from 48 4D worlds, where fixed code motion still conflicts with target appearance and source-color leakage reaches 64.17%.
Coding agents can already write Blender or MuJoCo scripts that lock layout, trajectories, collisions, and camera paths. The video model is then asked only to paint that low-fidelity draft. Physics is no longer its job.
A usable renderer has to clear two checks together. World preservation means the draft's geometry, motion, event order, and camera survive the prompt. Visual realization means colors and materials named in text stick to the right objects through occlusion, deformation, and viewpoint changes. Most video-to-video tests edit real footage that already has rich texture. Benchmarks such as PROWBench mainly check behavior against engine logs. A clip can track the motion and miss the paint, or nail the paint and redraw the forklift path. Older metrics mix those failures.
WorldDraft freezes 48 executable 4D world families and crosses two axes on the same dynamics, 3 by 3, for 432 paired cases. Each family is tagged by its main stress: multi-object layout, multi-stage events, occlusion and reappearance, camera motion, non-rigid deformation, or hybrid contact. Real videos are motion blueprints, not pixel targets. GPT-6-Astra turns interactions and cameras into MuJoCo or Blender scripts that log geometry, trajectories, event times, and camera poses. A family is kept only if the simulation stays stable, the interaction matches the reference, and the binding stress is unambiguous.
Granularity adds structure only. P0 is primitive shapes with flat shading. P1 is a normal simulation mesh with basic lighting, and it is the default operating point. P2 is previs: dense meshes and realistic shadows, still synthetic. Appearance changes the surface only. Neutral drafts are gray clay, so paint has to come from text. Aligned drafts agree with the prompt at that granularity. Conflicting drafts disagree on purpose, such as blue metal against a request for red plastic. Geometry, motion, and camera stay fixed. Each prompt names two to four checkable attributes, points to objects by role (the leading cube), and leaves out trajectories so the text cannot repair a sparse draft.
Scores are split. Preservation uses object counts, trajectory-matched box IoU, interaction success, event-time coverage, and camera rotation plus scale-normalized translation. P0 is not punished for detail that exists only in P2. Rendering fidelity is attribute-level yes/no questions. The judge sees the prompt and the output video, not the source draft. Persistence counts only when an anchor frame is correct and the same object keeps that attribute later. A wrong color that never changes scores zero. Conflict penalty is the fidelity gap between aligned and conflicting drafts. Source leakage is the share of conflicting cases that keep the source attribute. The reference cell is P1 with neutral appearance. Overall weights full-grid performance, granularity gains, and conflict robustness at 0.50, 0.25, and 0.25.
WorldDraft-Primer trains nothing. It edits the first draft frame and leaves every later frame untouched, so the recorded motion stays put. Qwen3.5 picks objects that are easy to drop. An open-vocabulary detector boxes them. Programmatic SVG draws corner brackets: monochrome brackets anchor structure, and target color, coarse texture, and soft shadows on the brackets and the ground anchor appearance. The original surface color stays inside the object, so a conflicting draft is still a conflict.
Headline scores average the full 3 by 3 grid over 12 models. The table keeps the extremes of preservation, fidelity, and leakage, plus the two best overall scores. Open models left out of the table land between OmniWeaving at 57.62 and Bernini-R at 62.97.
| Model | WP | RF | AP | Leakage | Overall |
| SolarRenderer-33B | 63.28 | 69.97 | 61.68 | 40.37% | 62.59 |
| LynnReal-Omni-32B | 60.52 | 75.51 | 68.41 | 30.54% | 64.89 |
| MiniMax-H3-33B | 50.70 | 76.38 | 70.32 | 28.64% | 63.63 |
| Wan 3.0 | 44.37 | 80.40 | 75.41 | 27.39% | 63.36 |
| Seedance 2.5 | 58.66 | 75.55 | 69.20 | 41.57% | 64.73 |
| Wan2.1-VACE-14B | 60.97 | 60.86 | 51.68 | 64.17% | 56.89 |
SolarRenderer leads preservation at 63.28. Wan 3.0 leads fidelity at 80.40 and persistence at 75.41, with the worst preservation at 44.37. Seedance 2.5 scores 64.73 overall, just behind LynnReal-Omni at 64.89, and still leaks source color on 41.57% of conflict cases. Across all 12 models, overall runs from 56.89 to 64.89, an 8-point spread.
Under neutral appearance, P0 to P2 raises structure for all 10 open-weight models. Wan2.1-VACE gains +18.82. Appearance and binding gains are negative for all 10. OmniWeaving-8.3B drops the most, appearance -14.47 and binding -15.95. The paper text says 9 of 10; Table 1 shows 10 of 10. Seedance 2.5 is the only model of the 12 with both gains positive, +2.51 and +3.21. Wan 3.0 barely moves: structure -0.35, appearance -0.58, binding -0.16.
At P1, conflict penalty peaks at 10.63 for OmniWeaving and bottoms at 0.53 for Wan 3.0. Source leakage peaks at 64.17% for Wan2.1-VACE. UniVideo-13B leaks 52.24%. Wan 3.0 and MiniMax-H3 still leak 27.39% and 28.64%.
On the warehouse forklift, after occlusion, spatial preservation for LynnReal-Omni and Seedance 2.5 is often at or below 18.8 and falls to 6.2. Camera and shelves are replaced by a view invented from the prompt. On a 3 by 4 sailboat fleet, both models scramble colors under a neutral draft. From P1 to P2, LynnReal-Omni's realization falls from 73.1 to 46.2 when the draft agrees with the text, and to 17.3 when it conflicts. It also adds an extra cluster of green boats.
Primer is reported on two models. OmniWeaving moves from 61.61 to 64.60 in preservation, 66.37 to 70.93 in fidelity, 50.97% to 47.55% leakage, and 57.62 to 60.13 overall. Text prompt editing reaches 58.52. MiniMax-H3 moves from 76.38 to 80.21 in fidelity and from 63.63 to 65.56 overall, ahead of LynnReal-Omni at 64.89 and Seedance 2.5 at 64.73. Text editing on MiniMax-H3 lands at 64.89, the same overall as untouched LynnReal-Omni.
Once code locks the motion, the video model is a conditional video-to-video step. Denser previs holds the kinematics and overrides the requested paint, which is why P1 is the measured operating point. A sharp preview in the wrong color is riskier than gray clay. Leakage from about 27% to 64% means a placeholder surface with text in charge is not an interface you can trust yet.
Primer is a test-time patch. On two architectures, preservation and fidelity rise together. Text editing does not do that. The overall lift is about two points, and leakage stays above 27%. The full leaderboard spans eight points. A larger renderer has not separated the two goals.
Primer is missing from SolarRenderer, the preservation leader, from Wan2.1-VACE, the leakage leader, and from both commercial systems. Whether a first-frame cue moves a model that already sits at one extreme is untested.
The granularity penalty remains. With Primer, OmniWeaving's P0-to-P2 appearance gain goes from -14.47 to -15.45, binding from -15.95 to -17.02, structure only another +0.76, and conflict penalty from 10.63 to 10.41. Absolute scores rise. Finer drafts still hurt the paint.
Fidelity comes from yes/no questions whose judge is unnamed, so overlap with the video models under test is unknown. The paper states that FVD does not measure these two goals, and the custom scores should not be lined up against ordinary video leaderboards. GPT-6-Astra translated the 48 worlds from real clips, so blueprint choice and translation error are baked into what preserving the world means. Prompts carry only two to four attributes. A spatial score at or below 18.8 is one case, not the benchmark-wide preservation number. Primer also trusts a detector. A bad box anchors the bracket to the wrong object, and no miss rate is reported.