Addressable Memory for Video World Models
Xindi Wu, Sven Elflein, James Lucas, Olga Russakovsky, Laura Leal-Taixé, Despoina Paschalidou, Jonathan Lorraine, Aljoša Ošep
cs.CV, cs.LG
2026-08-08
Training-free WorldTrace gives compressed KV-cache slots in-distribution virtual positions, lifting long-rollout consistency +15.5% and recall +19.5%.
Interactive video world models generate footage frame by frame while responding to input, like playing a game. They keep generated frames in a KV cache as visual memory and roll forward autoregressively. Once generation runs past the length seen in training, that memory starts to fail. These models use RoPE (rotary positional embeddings, which tag each frame with an angle that rotates over time): the further out in time, the larger the angle. Past the training range, the angle lands in a region the model has never seen, and attention can no longer read those old frames reliably.
The second trap is subtler. Compressing the cache means collapsing many frames into one slot, and the obvious move is averaging keys. But averaging in the RoPE-rotated space mixes keys pointing in different directions, and they partially cancel. This is phase cancellation, and it corrupts the compressed memory regardless of what was stored.
WorldTrace keeps compressed memory both findable and informative, with no retraining. It splits the attention window into a recent stretch kept verbatim and a set of summary slots holding compressed history.
The first move is virtual positioning. Each summary slot gets a timestamp pinned at a fixed offset behind the current query, inside the range seen during training. No matter how long generation runs, every slot stays in distribution and individually addressable.
The second move is compressing in canonical space to dodge phase cancellation: unrotate each source key back to its canonical orientation, average there, then re-rotate at the virtual position's angle. The paper shows that a query attending to the resulting slot scores about what it would have scored attending to the source frames on average, so information is preserved.
On top of this addressable cache, the authors study two compression schemes:
The authors also release LoopBench: the model traces a path, returns to a starting point A, and the compressed cache is scored on whether it can reconstruct the original scene, measured by Position-Aligned CLIP similarity (PAC). Experiments run on Matrix-Game-2 (MG2-1.3B, a 1.3B distilled autoregressive game world model built on Wan 1.3B, with a training context of only 6 AR chunks).
| Setting | Method | Result |
| N=48 TempSSIM | Sliding window (baseline) | 0.472 |
| N=48 TempSSIM | WorldTrace-Field | 0.545 (+15.5%) |
| LoopBench ABA PAC (N=16) | Sliding window | 0.723 |
| LoopBench ABA PAC (N=16) | WorldTrace-Landmark | 0.864 (+19.5%) |
The ablation isolates both mechanisms. Swapping naive averaging for canonical averaging cuts LatentDiff by 25.3% at Ns=4 (0.312 to 0.233); naive averaging's error grows with more slots, while canonical averaging does not. On LoopBench, WorldTrace-Landmark beats the sliding window on PAC across all four difficulty tiers (topology, length, camera pan, multi-revisit), and the longer the detour the bigger the gap (ABA from N=8 to N=32: sliding window falls 0.859 to 0.627, WorldTrace 0.922 to 0.825). The lead narrows at the hardest 360 degree pan.
Video world models are a candidate path to playable, interactive virtual worlds, but long-horizon consistency has been stuck on the memory mechanism. WorldTrace's selling point is being training-free: it drops onto an existing temporal-RoPE autoregressive model and pushes generation from seconds toward minutes. For anyone building game world models or embodied simulation environments, this is a cheap engineering lever that leaves the training pipeline untouched.
It also surfaces an overlooked failure mode. KV-cache compression is not just averaging, and phase cancellation in RoPE-rotated space is a silent bug that propagates downstream. That lesson applies to any long-context RoPE system that compresses its cache.
The authors draw the boundary themselves. WorldTrace applies only to autoregressive models with temporal RoPE and a fixed cache budget, not to every world model architecture. Field blurs specific detail; Landmark depends on detecting scene entries correctly, and a missed landmark does nothing useful. Both projections are fixed and non-adaptive, which may fall short in interactive settings with free movement and frequent viewpoint changes. Even with extended rollouts, generation still degrades at very long horizons.
One caveat when reading the numbers: LoopBench's PAC is a CLIP similarity, and CLIP is semantic rather than sensitive to fine geometry or texture. The +19.5% episodic recall means scenes are recognized at the semantic level, not necessarily pixel-level reconstruction. All evaluation is on a single model, MG2-1.3B, so generalization to larger models or other world models is unverified.