Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation
Hmrishav Bandyopadhyay, Xuanchi Ren, Zijian Huang, Jay Zhangjie Wu, Tianshi Cao, Ruilong Li, Bryan Chu, Sanja Fidler, Yi-Zhe Song, Zian Wang
cs.CV
2026-08-13
CMD replaces the bidirectional distillation teacher with a causal one that cannot see future frames or camera controls when scoring each target. Camera rotation error drops to 1.27 on the hard split, VBench-I2V total reaches 88.46, and the camera-motion score nearly doubles versus the best autoregressive baseline.
Autoregressive video models generate block by block, which suits streaming and interaction, but each denoising step is expensive, so pipelines distill them into few-step students via distribution matching distillation (DMD). The supervision is where things break: DMD needs a teacher to score the student's outputs, and existing pipelines use a bidirectional teacher that scores complete clips. The score for frame t can then depend on frames and controls after t, information the online student never had. The paper calls this the teacher-student context mismatch. Camera control suffers most, since the teacher sees the whole camera trajectory while the student must execute each pose update online; long rollouts suffer because treating each local clip as self-contained discards the rollout context that produced it.
CMD has three pieces. First, train a causal teacher: fine-tune a pretrained bidirectional model with a Diffusion Forcing objective (independently noised frames denoised under noisy histories), yielding a multi-step causal model whose attention cannot reach future frames or controls. Second, initialize the few-step student directly from those weights, skipping expensive ODE-matching or consistency-distillation warm-up; during distillation the student rolls out exactly as at inference while the frozen causal teacher scores it, so teacher training, distillation, and inference share one causal formulation. Third, Prefix Scoring conditions each score on the cached student-generated prefix that actually produced the target, instead of a sequence of noised DMD targets; a block-causal attention mask scores all targets in a single teacher pass. Prefix Corruption adds a small amount of independent noise to the prefix to blunt artifacts from unreliable early rollouts, with a frame-dependent schedule for long videos. Cameras are encoded as frame-relative pose increments rendered into a ray map, and the teacher for frame t may only consult controls up to t.
Base model is Cosmos-Predict2.5-2B:
| Benchmark | Metric | Best baseline | CMD |
| VBench-I2V short | Total | 87.63 (Causal Forcing) | 88.47 (chunk 4) |
| VBench-I2V | Camera Motion | 42.54 (LingBot-World) | 76.12 (chunk 4) |
| SANA-WM long, 501 frames | Total | 80.72 (Context Forcing) | 81.39 (chunk 1) |
| SANA-WM long | Dynamic Degree | 51.25 | 77.50 (chunk 4) |
| SANA-WM camera, Simple | rotation err. | 1.2104 (SANA-WM) | 0.8601 (chunk 4) |
| SANA-WM camera, Hard | rotation err. | 2.0453 (SANA-WM) | 1.2718 (chunk 4) |
The most informative ablation row: swap the causal teacher back for a bidirectional one within the same framework and short-video Total falls from 88.33 to 82.65, so most of the gain is the causal alignment itself. A blind Gemini 3.1 Pro judge prefers the chunk-4 model over six baselines in 64% to 88% of 100-trial pairwise runs.
Few-step autoregressive generation is the real-time path for interactive world models, and this paper turns "the teacher must not peek at the future" into a cheap, reproducible recipe: one causal teacher serves both scoring and student initialization, and the same bounded-prefix formulation extends to long videos without widening the teacher's attention window. The camera ablation isolates the mechanism cleanly: with the same ray-map conditioning, Simple-split rotation error is 5.73 under a bidirectional teacher and 1.37 under full CMD. When a student disobeys controls, the culprit is often a teacher that was taught with future information.
On long videos, subject and background consistency fall below Rolling Forcing and others (88.59 vs 96.09); dynamism and consistency look traded off, and more motion is not the same as a steadier picture. In the camera ablation, Base CMD without Prefix Scoring achieves lower Hard-split rotation error (2.16) than the full method (2.56), so Prefix Scoring's contribution is not monotone in that setting. Perceptual results rest on a single LLM judge with a hierarchy authored by the authors. No inference latency or frame-rate numbers are given, so the actual speed of the few-step student cannot be checked from the paper. Only a 2B backbone is evaluated.