History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models
Hangyi Kuang, Chengzhi Wang, Yefei He, Weijie Wang, Yuke Zhao, Zhuokun Chen, Zisu Huang, Yifan Yang, Xiaoxuan He, Feng Chen, Bohan Zhuang
cs.CV
2026-11-02
History Forcing corrects few-step video distillation with a history residual. A 1.3B model scores 85.07 at 15s and 79.82 at 60s; object consistency rises from 58.43 to 62.18.
Few-step autoregressive video diffusion writes the clip one chunk at a time, a short block of latent frames. Each block is cheap. Over a long rollout the subject and the layout still drift, because generation error accumulates. After a person or object leaves the frame, the latest chunks say nothing about it. When the camera returns, the model often draws a different version. World models make the gap sharper: object permanence depends on observations from much earlier than the recent context.
DMD, distribution matching distillation, pushes a few-step student toward a pretrained diffusion teacher by matching scores. A bidirectional teacher supplies a strong quality signal and never sees distant autoregressive history. Distilling an autoregressive teacher that does see that history ties consistency and image quality to one conditional distribution. Visual faults in the autoregressive teacher transfer to the student. That failure mode is history-quality coupling.
History Forcing changes the distillation target and leaves the student's sampler alone. A frozen autoregressive teacher reads the same noisy chunk twice. The only change is whether extra history is attached. Recent context stays fixed. The gap between the two score or velocity predictions is the context residual, the direction in which that history pushes the current chunk. Scale it by λ and add it to the original target.
Long-video training still uses DMD. The reference score comes from a bidirectional Wan2.1-T2V-1.3B teacher. The correction is λ times the history residual, the same shape as classifier-free guidance, applied to history rather than to the prompt. The student stays close to the bidirectional teacher and does not have to match the autoregressive teacher's full distribution. The default is λ = 1, active only at diffusion time t < 750, the lower-noise part of the schedule.
Extra history comes from earlier chunks of the student's own rollout, before drift has piled up. Both teacher passes share the latest 6 latent frames. The history-conditioned pass also gets the 12 frames just before that block, so the residual tracks earlier observations rather than the recent frames themselves. Each update supervises one randomly chosen chunk. The student starts from Causal Forcing's causal ODE checkpoint and trains for 300 steps under the Self-Forcing++ schedule. Rollout length is sampled from 5 to 15 seconds, and the DMD loss is applied to the last 5 seconds. The autoregressive teacher was trained on 5-second clips. The main run fine-tunes it on 10-second clips from UltraVideo-Long.
Off-screen change is not in those old student frames. A teacher starts from the same image and holds the camera at the initial pose, rendering the interval the student never sees. A few-step network is a poor score model at arbitrary noise levels, so the loss switches to velocities from an EMA teacher, following OPSD-V, with no extra network fit to student samples. At λ = 1 the target is exactly the history-conditioned velocity. That setting is the adapted OPSD-V control. The reported world-model run uses λ = 3. The backbone is DreamX-World-5B, LoRA-tuned for 100 steps, with the loss on all 4 denoising steps of revisit chunks. The set is 200 starting images, each with 4 leave-and-return camera paths.
On 15-second VBench, the matched baseline without history supervision scores 84.44 on quality. The residual from the 5-second teacher raises that to 84.82. Fine-tuning the teacher on 10-second clips reaches 85.07. Infinity-RoPE scores 84.54. LongLive and Rolling Forcing score 84.20.
Consistency does not lead the table. Subject consistency is 93.21 and background consistency is 95.03, against 95.61 and 95.80 for Rolling Forcing. Dynamic degree is 70.00, against 51.11 for Rolling Forcing and 76.67 for the no-history baseline. Against that baseline, appearance is steadier and motion is smaller.
The 60-second score is outside the training horizon. Rollouts stop at 15 seconds. Evaluation generates 60 seconds and gets a VBench-Long aggregate of 79.82, above Rolling Forcing at 79.22 and LongLive at 79.12. Both of those 1.3B students use a 14B DMD teacher. History Forcing uses a 1.3B bidirectional teacher. Subject consistency is 97.32 and background consistency is 96.19, best in the 1.3B group. The 5B LongLive-2.0 model scores 79.90. Dynamic degree is 49.06, below Causal Forcing at 72.32. Aesthetic quality is 58.06, below Rolling Forcing at 63.50. The other 60-second numbers are taken from the LongLive-2.0 report.
| Setting | Method | Metric | Result |
| 15s VBench | History Forcing, 10s teacher | Quality Score | 85.07 |
| 15s VBench | matched baseline | Quality Score | 84.44 |
| 15s VBench | Infinity-RoPE | Quality Score | 84.54 |
| 60s VBench-Long | History Forcing, 1.3B teacher | Aggregate | 79.82 |
| 60s VBench-Long | Rolling Forcing, 14B teacher | Aggregate | 79.22 |
| 60s VBench-Long | LongLive-2.0, 5B | Aggregate | 79.90 |
| PlayWorld out-of-sight | History Forcing | Evolution Quality | 2.70 |
| PlayWorld out-of-sight | DreamX-World-5B | Evolution Quality | 2.35 |
| MemoBench Real | History Forcing | Object Consistency | 62.18 |
| MemoBench Real | baseline | Object Consistency | 58.43 |
On PlayWorld's out-of-sight subset, 43 clips on a 1-to-5 scale, evolution quality moves from 2.35 for DreamX-World-5B and 2.42 for OPSD-V to 2.70. Subject identity moves from 2.70 and 2.72 to 2.98. On MemoBench Real, 164 clips, object consistency moves from 58.43 to 62.18. Eight of nine metrics beat OPSD-V at λ = 1. SSIM does not: 0.5463 versus 0.5487. PSNR moves from 14.297 dB to 14.467 dB.
With the 5-second teacher, raising λ from 0 to 2 lifts subject consistency from 91.84 to 93.29. Quality peaks at λ = 1 (84.82), not at λ = 2 (84.50). Dynamic degree is 89.72 at λ = 0.5 and 73.61 at λ = 2. Removing the shared 6 frames drops quality from 84.82 to 81.91. If the history term is not normalized jointly with the DMD gradient, dynamic degree falls to 59.72. Joint normalization restores it to 78.06, while subject consistency drops from 93.78 to 92.58. Training is 2.31% slower per step than DMD and uses 1.99% more memory.
On a pipeline that already runs DMD or OPSD, this is a change to the target. Quality supervision stays with the bidirectional teacher or the EMA teacher. Long-range history enters only as a residual.
A student trained against a 1.3B teacher beats same-size students distilled from 14B teachers on the 60-second aggregate, after rollouts of at most 15 seconds. If that gap holds under one evaluation protocol, the long-horizon miss looks more like an under-weighted history term than a teacher-size problem. The out-of-sight half uses privileged video at training time. The fixed camera sees evolution the student misses. At inference that view is gone, so the change has to sit in the weights.
The gains are incremental. Fifteen-second quality is 0.53 above Infinity-RoPE. The 60-second aggregate is 0.60 above Rolling Forcing and 0.08 below a 5B model.
Long-video experiments stop at 1.3B teachers and students. The world model is a 5B backbone. Historical supervision cannot outrun the base model's grasp of temporal dynamics. If the fixed-camera teacher barely moves the scene, the unseen interval has no signal to distill.
The 60-second comparison quotes LongLive-2.0 and does not re-run those systems. Prompt sets and score normalization cannot be checked from this paper. The 15-second quality score leads. Subject and background consistency do not. The aggregate includes dynamic degree.
Evolution quality of 2.70 is still low on a 1-to-5 scale. MemoBench PSNR stays near 14.5 dB, and SSIM near 0.55. At λ = 5, MemoBench object consistency rises to 62.78 while PlayWorld evolution quality falls to 2.65. λ, the noise threshold, and gradient normalization all trade consistency against motion. In the 5-second-teacher ablation, a threshold of 500 scores 85.08, above the default threshold of 750 at 84.82. The headline 85.07 uses the 10-second teacher and the default threshold, so the two figures are not one knob turned two ways. There is also no test of a privileged teacher that renders the wrong off-screen state.