LayerRecall: ZJU & HKU fix long-video amnesia with selective memory routing
新智元 · wechat · 2026-09-06
Autoregressive video models forget identities and props when characters leave and return, because KV caches prioritize recent frames. ZJU/HKU propose LayerRecall, routing retrieved history K/V only into memory-sensitive DiT layers (what to retrieve + where to use), trained via Cross-Horizon Prediction Matching with a 384-frame teacher supervising a 32-frame student—only 1.65M trainable params on a frozen 5B backbone. Results: MemoBench Overall 0.548 (best baseline 0.531), MovieBench 0.578, VBench-Long parity, high-frequency flicker power ratio 0.60→0.38, 3.5s overhead. The router transfers zero-shot to LongLive and Self-Forcing. 6.6MB weights, training code and 100 eval prompts open-sourced.
More from Multimodal
- Generative AI point-and-click detective game built without a multimillion-dollar budget — tristanbob · 2026-09-06
- A sugar cube containing a full kitchen: the impossible-geometry video prompt, in full — umesh_ai · 2026-09-06
- Redditor Shares an AI-Generated Action Short Film — Ok-Vegetable-2455 · 2026-09-06
- Astra recreates Agamemnon's helmet in 3D, outputting GLB and Blender files in 9 minutes — eigenron · 2026-09-06
- 800 photos recreate a 220-year-old Japanese inn in 3DGS using LichtFeld exposure correction — janusch_patas · 2026-09-06
- Swapping Faces in Hailuo Videos: One Character Works, Two-Character Fights Fall Apart — MIRIVUM · 2026-09-06