RECAP-Forcing: Retaining Content Appearances for Long Video Generation
Haiyang Xu, Zheng Ding, Zhuowen Tu
cs.CV
2026-08-27
Training-free RECAP-Forcing indexes KV by appearance novelty. On Self-Forcing, 60s VBench-Long Total goes 75.9 to 79.5 and Dynamic Degree 27.5 to 58.1.
Causal video models roll forward in blocks, but the attention window is finite. Distilled generators such as Self-Forcing only keep KV for the last few frames. Everything older is dropped or squeezed into a coarser summary.
Memory organized by time sits on a bad trade. Drop early appearance and identity drifts. Pin the opening frames too hard and later motion freezes. Deep Forcing, MemRoPE, and Rolling Sink all stay on that axis: widen the sink, compress evicted keys, or replay the first seconds. They assume that rendering a subject at t=60s means walking back along the timeline to when it was last seen.
RECAP-Forcing indexes memory by appearance novelty. A subject walking in, a disoccluded region, a newly panned storefront: those are novel appearance events. The method copies the patch KV only at those moments. Patches that optical flow can still trace are not stored again. Capacity grows with new content, not with duration. A person who entered at second three remains directly addressable at minute five, from the original uncompressed KV.
Two mechanisms share that rule.
The whole thing is an inference hook with no new parameters. The main setup sits on Self-Forcing distilled from Wan2.1-T2V-1.3B: 832×480, 3 latent frames per block, a 6-frame window (1 sink + 2 prior + 3 current), 240 latent frames for 60 seconds. The same hyperparameters attach to Infinite-Forcing, LongLive, and Helios 14B.
Evaluation is VBench-Long, 128 prompts × 5 seeds, 60 seconds. Scores are percentages.
| Method | Dynamic Degree | Semantic | Total |
| Self-Forcing | 27.5 | 58.0 | 75.9 |
| unreinforced sink | 30.4 | 62.3 | 76.8 |
| RECAP-Forcing | 58.1 | 65.5 | 79.5 |
| Infinite-Forcing | 61.3 | 62.9 | 78.3 |
| Ours | 71.3 | 65.4 | 79.7 |
| LongLive | 20.3 | 65.9 | 78.1 |
| Ours | 26.6 | 66.8 | 78.5 |
| Helios 14B | 36.2 | 45.4 | 70.3 |
| Ours | 47.8 | 60.5 | 76.3 |
On Helios, Object Class jumps from 36.1 to 83.7, the largest semantic gap closed. LongLive is already stable; Dynamic Degree only moves 20.3 to 26.6 and Total gains 0.4.
Against other training-free methods on Self-Forcing: Deep Forcing 76.2 Total / 32.0 Dynamic Degree; MemRoPE 78.7 / 41.2; Rolling Sink 78.9 / 23.0. RECAP-Forcing is 79.5 / 58.1. Rolling Sink looks stronger on consistency metrics and stiffer on motion.
Component split on Infinite-Forcing: reinforce the sink alone and Dynamic Degree falls 61.3 to 41.6 while Semantic rises to 65.2. Bank alone lifts Dynamic Degree to 78.2 with a small consistency drop. Both together reach 71.3 Dynamic Degree and the best Total, 79.7. The sink holds the opening cast; the bank holds mid-video arrivals.
λ=5 is the peak, with about 24% of attention on the sink. λ=8, 10, 12 drop Total to 79.2, 78.9, 78.6. Bank size from a quarter frame to four frames barely moves Total (79.6-79.8); eight frames falls to 79.3. One frame is the default.
Dynamic Degree rises while mean flow falls. Camera flow goes 1.20 to 0.90, object flow 0.71 to 0.66. Erratic global drift is suppressed and near-frozen clips are pushed over the genuine-motion threshold; the drift tail shrinks from 37 videos to 24.
The appendix adds identity numbers. On 20 prompts where the subject stays on screen, DINOv2 CLS similarity from an early window to the last 10 s: 0.745 to 0.850 on the full frame, 0.642 to 0.696 on a subject crop. At five minutes the recency baseline ends at 0.61; RECAP-Forcing plateaus at 0.81 from minute one through minute five. A constructed leave-and-return (7.5 s empty gap, 5× the attention window, sliding cache cleared at cuts) lifts full-frame similarity from 0.699 to 0.752.
Nine raters, 216 blind pairs. Overall win rate versus Self-Forcing is 84.3%; versus the three competitors 77.8%, 75.0%, 79.6%. On a GH200, one minute costs 185 s versus 288 s, about 1.6×, and that factor does not grow with length.
For anyone already running a causal video generator, this is a plug-in at inference. The transferable claim is the indexing axis: a finite cache should be spent on what newly appeared, not on how long ago it was seen. Minute-scale consistency then sits in the memory layout, instead of being reconstructed from a compressed past.
The generator is not retrained, and there is no learned object slot. The unit is a patch at first appearance, not a tracked entity. That is why it is cheap, and why the failure modes below are unsurprising.
Optical flow repeatedly flags rain, spray, and rippling water as novel, filling slots with texture that later frames cannot reuse. Longer-range correspondence or semantic grouping is named as future work, not measured.
Gains are uneven. LongLive's Total only rises 0.4. On Self-Forcing, VBench Subject Consistency slips 98.0 to 97.7 even as motion rises. The reappearance test cannot be written as a prompt: current backbones ignore staged enter/exit instructions, so the authors cut segments and clear the window at inference. That tests whether the bank can carry appearance across a gap, not whether the model follows a plot.
The 1.6× wall-clock is real. Admission runs RAFT-small at half resolution with 12 refinements.