Training Object Permanence in World Models
Haotian Zhang, Fengyuan Yu, Dezhi Luo, Haoran Sun, Zehong Zhao, Qingying Gao, Yihan Li, Siyuan An, Huayi Qin, Yilan Zhang, Zhengze Jiang, Pinyuan Feng, Renrui Zhang, Ziyu Guo, Letian Wang, Mengyue Yang, Kangfu Mei, Maijunxian Wang, Ran Ji, Vikash Kumar, Freda Shi, Chandra Sripada, Vincent C. Muller, Philip Torr, Alan Yuille, Nikolaus Kriegeskorte, Felix Juefei-Xu, Lvmin Zhang, Jieneng Chen, Yilun Du, Hokin Deng
cs.AI, cs.CV
2026-09-24
WROP builds 1.5M Blender clips across 150 cognitive tasks; PWM-WROP, a 16B fine-tune, ranks first among continuations (Elo 1679.5) and third of 14 models in a blind study.
Video generators look photorealistic until an object ducks behind an occluder, phases through a wall, or hangs in mid-air after its support is pulled. Object permanence and solidity are core-knowledge priors that human infants acquire within months: unseen objects still exist, solids do not interpenetrate. Most video-model evals stay in 2D or image-to-video, and they lean on VLM judges. Those judges themselves lack the same core knowledge, so they are a poor referee for the skill under test.
WROP turns the two priors into a training and exam stack: 150 hand-authored Blender generators, six task families, 1.5 million training clips, and a fixed 300-question exam. The same corpus is used to fine-tune PWM-WROP, a 16B continuation model, to test whether the priors can be trained in rather than hoped for as an emergent side effect.
Six families split across the two priors. Permanence: a moving object reappears after occlusion with identity intact; a static layout survives a moving screen; a hidden object stays bound to a moving container. Solidity: an aperture either blocks or admits based on size; support removal triggers an immediate fall; a collision conserves count and sends objects on diverging paths. Every generator places the key physical event at or after frame 60, so the first half is context and the second half is the prediction target, matching a video-to-video contract.
Trajectories are keyframed by hand, not solved by a rigid-body engine, so occlusion and contact land on controlled frames. Surface parameters (color, material, lighting, camera) are randomized; structural parameters (object count, path, aperture ratio) vary systematically so a model cannot cheat on texture. Each sample is 120 frames at 1280x720 and 24 fps, plus a prompt, per-frame poses, and scene metadata.
Fourteen models are grouped by interface. True continuation extends the clip from its last frame. Reference-to-video treats the input as a visual hint and regenerates the event on its own timeline. Edit/transfer repaints the source span and therefore cannot depict events that start after the occlusion boundary. Primary metric is a blind pairwise study with 20 qualified raters, fitted by Bradley-Terry to an Elo scale with mean 1500. Pixel metrics are secondary: an editor that faithfully repaints a static pre-event scene can score high SSIM without ever showing the object re-emerge.
PWM-WROP fine-tunes NVIDIA Cosmos3-Nano for one epoch on the 1.5M clips. Architecture and tokenizer stay frozen. Training geometry is 320x192 with 57 conditioning frames and 60 predicted frames. The PWM training stack on AWS Trainium2 is released with the weights.
361 decisive pairwise judgments (about 50-52 games per model). The two reference-to-video systems Wan 3.0 Prime and MiniMax H3 tie for first at Elo 1723.6 (77.9% score rate). PWM-WROP sits third overall and first among continuations at 1679.5 (73.1%), about 223 Elo above the next continuation, Grok Imagine video-extend at 1457.0. Edit models cluster in the 1470-1520 band.
| Model | Interface | Elo | Score rate |
| Wan 3.0 Prime | Reference-to-video | 1723.6 | 77.9% |
| MiniMax H3 | Reference-to-video | 1723.6 | 77.9% |
| PWM-WROP | True continuation | 1679.5 | 73.1% |
| Seedance 2.5 | Reference-to-video | 1649.6 | 69.6% |
| Grok Imagine | True continuation | 1457.0 | 44.2% |
| MAGI-1 24B | True continuation | 1248.0 | 19.2% |
By family, PWM-WROP ranks first on static occlusion (OP-2) and top-three on the other two permanence families. Solidity is mixed: second on drop, fifth on obstruction, eighth on collision. Reference-to-video systems pick up much of their lead on solidity, where regenerating the whole scene is easier than extending the last frame.
At a matched 320x192 evaluation scale, PWM-WROP posts the best LPIPS (0.081 vs next 0.105) and MS-SSIM (0.921 vs next 0.877). That is despite a native 320x192 output against 720p and 1080p competitors. The paper treats these numbers as a visual-consistency check, not a permanence score.
This is a train-and-exam stack for object permanence that does not outsource judging to a VLM. Interface class explains the leaderboard better than parameter count: editors cannot leave the input interval, reference-to-video systems can restart the timeline, and only true continuation has to keep physics going from the last frame. PWM-WROP changes the training signal, not the architecture, and moves continuation Elo from MAGI-1's 1248 to 1680. For anyone building world models, occlusion and contact are unit tests, not capabilities that appear for free at scale.
Motion is authored as keyframes, not a physics engine, so a "plausible" collision is not a conserved-momentum collision. PWM-WROP is the only model trained on WROP, so the gap versus the other 13 systems is not a clean data ablation. Reference-to-video winners may redraw object identity and layout rather than preserve them. Elo intervals are wide (PWM-WROP [1604, 1782]); the top four are not cleanly separated. Solidity, especially collision, responds less to this fine-tune, which fits the idea that occlusion tracking and contact dynamics are different internal representations. The exam has 300 items and only four items per model pair, so family-level ranks are noisy.