LOCI: Spatial Linear Memory for Streaming World Models
Ji Xia, Tingting Liao, Xuezhi Liang, Hao Li, Guangyi Liu
cs.CV
2026-10-01
LOCI mixes camera-conditioned linear memory with KV attention in a 5B world model: +0.89 dB revisit PSNR over full softmax at equal budget, 300 s streaming at constant memory.
A camera-controllable video world model has one unforgiving requirement: when the camera returns to a place it saw minutes ago, the scene has to come back with it. That takes two things at once, remembering past observations and retrieving the right one for the current viewpoint.
The two standard memory designs each fail one half. Full softmax attention over history keeps every KV pair, so detail survives, but storage grows with video length and the same-recipe baseline here runs out of GPU memory at 157 seconds. Recurrent memory (linear attention, state-space models) compresses history into a fixed-size state, which is compact but leaves individual past observations inaccessible. Retrieval-based selection cuts the budget but can discard evidence that matters later. LOCI, from a MBZUAI team, keeps both traces and makes them talk to each other.
The base is the 30-block Wan2.2-TI2V-5B video transformer, adapted for chunk-causal generation with five latent frames per chunk. Fifteen blocks are hybrid: intra-chunk softmax attention plus a recurrent linear-attention memory. The other fifteen keep historical softmax attention over explicit KV.
History access has two modes: dense, or bounded sparse with one sink frame, a panorama bank of up to 20 frames, the 8 most recent frames and the current chunk, at most 34 latent frames total. The bank updates by field-of-view coverage, favoring earlier observations with complementary viewpoints; discarded frames are gone for good, but their content persists in the recurrent state. Training uses chunk-wise diffusion forcing for 5,000 updates; the recurrent branch adds 90.7M parameters (1.7% of 5.38B).
On the MIND benchmark (a recorded segment is context; the generated continuation is scored against the recording), LOCI is evaluated zero-shot and still beats GIM-World, which was trained on MIND, on MSE, PSNR and SSIM, with LPIPS slightly behind (0.643 vs 0.630):
| Model | PSNR↑ | SSIM↑ | LPIPS↓ |
| GIM-World (trained on MIND, reported) | 13.40 | 0.414 | 0.630 |
| HY-WorldPlay (8B, run by the authors) | 12.54 | 0.398 | 0.715 |
| LOCI (zero-shot, full history) | 14.36 | 0.464 | 0.643 |
| LOCI (bounded sparse) | 13.02 | 0.409 | 0.679 |
That is 1.82 dB above the strongest model the authors ran. The controlled comparison (same backbone, data, recipe, 5,000 updates) isolates the architecture: under an identical bounded KV budget the hybrid gains +0.89 dB PSNR (95% interval 0.64 to 1.15) and wins 44 of 50 segments, so the gain does not come from storing more history. On held-out Unreal Engine trajectories it leads by 0.62 dB at revisit instants; on four trajectories returning to places first seen over 60 s earlier, by 0.68 to 1.93 dB.
Cost: with full history, peak memory at equal length drops about 30% and reachable duration extends from 157 s to 225 s. In bounded mode it generates 300 s at a constant 23.6 GiB (full softmax under the same access: 27.6 GiB) at equal speed, 5.4 s per video second on an H200.
The analysis backs the mechanism: camera yaw is linearly decodable from the recurrent state (held-out R² = 0.946), co-visible tokens are about 0.1% of the history yet draw 11.1% of history attention (115× enrichment vs 108× for full softmax), and zeroing the readout at inference lowers that enrichment, so downstream queries do consume its content.
Streaming long-form generation is bottlenecked on memory, and this is an engineerable answer: detail in KV, summary in a fixed state, camera geometry as the index. The components (KDA, PRoPE, a Wan backbone) are off the shelf and the recurrent branch costs 1.7% extra parameters. Constant memory plus revisit consistency are exactly the two properties that decide whether a world model can stream. The gains stay under 1 dB, a structural incremental improvement, but the controlled experiments are clean and the mechanism analysis goes beyond reading a leaderboard.
The authors' own list: held-out trajectories are all Unreal Engine renders with no real-world captures; absolute fidelity is low for every model (benchmark-average PSNR below 15 dB), so the gains are relative; one 5B backbone with a short 5,000-update fine-tune; and the per-token vs per-chunk retention comparison uses a metric defined after the fact, which they flag as exploratory.
Questions that remain: all 30 blocks keep an explicit-history camera-attention branch, so the recurrent path halves only main-attention KV, and the 30% saving is not half the footprint. The external-model comparison on MIND is a reference under a unified protocol, not a controlled one, since models differ in size, sampling steps, and how they ingest the memory segment. Training builds the recurrent state from noised features while inference commits it from generated chunks, a mismatch the paper does not quantify separately.