NVIDIA's WorldTrace: Training-Free Memory Stops Video World Models From Forgetting Scenes
cindy_x_wu · x · 2026-11-21
The Problem
Interactive video world models generate realistic worlds but lack visual persistence: turn the camera away and back, and the scene has changed. The culprit is the KV cache acting as growing visual memory—once rollouts exceed the training horizon, temporal RoPE offsets go out of range and attention fails to retrieve stored content. Naively compressing the cache in RoPE-rotated space also degrades memory by averaging keys with incompatible positional phases.
Method: WorldTrace
- A training-free memory framework—no generator retraining needed
- Keeps compressed memory addressable by assigning each summary slot a distinct, in-distribution virtual position
- Merges keys only after removing their rotation, avoiding phase averaging
- The model's own attention decides which observations to keep verbatim vs. merge into compressed summaries, all within one bounded addressable memory bank
Results
On LoopBench, WorldTrace improves average DINOv2 recall by 32.6% over a sliding window at the same attended-cache size.
Authors span NVIDIA, Princeton, Toronto, and Vector Institute (first author Xindi Wu).
More from Research
- Looped transformers: more looping boosts reasoning but with diminishing returns past 3 passes and fewer CoT tokens — nrehiew_ · 2026-11-21
- Team releases TML-d24-varloop-research on Hugging Face to study looping's effect on CoT monitorability — nrehiew_ · 2026-11-21
- Loop only the middle layers: design detail shows 1.51-1.91x compute-equivalent gain vs 1.24-1.36x for last layers — nrehiew_ · 2026-11-21
- New Company Agent Bench: Gemini Neon leads at 87.4%, simple API harnesses beat products — philhchen · 2026-11-21
- NEJM Analysis of $45B in Medicare Payments: Pathology 99% AI-Exposed, Dermatology 7% — HealthcareAIGuy · 2026-11-21
- DeepMind spins up a math horizon repo amid unverified claims of Gemini solving open math problems — kimmonismus · 2026-11-21