Video diffusion models break physics because RoPE locks motion paths early, arXiv study finds

udmrzn · x · 2026-09-24

A new arXiv paper presents the first interpretability study of motion planning in text-to-video diffusion models, explaining why SOTA systems often violate physics.

Key findings:

Motion follows a 'first shape, then details' pattern: trajectories are committed during early denoising steps.
A specific subset of attention heads, identified via cross-attention trajectory patterns and causal head contributions, drives motion planning.
Rotary Position Embedding (RoPE) causes excessive spatial attention decay, prematurely locking candidate regions into physically implausible positions and suppressing coherent motion in adjacent frames.

Fix: a lightweight architectural modification that scales RoPE frequency across denoising steps. Both training-free and training-based experiments confirm improved physical consistency, without external priors or specialized data.

Original post →

Video diffusion models break physics because RoPE locks motion paths early, arXiv study finds

More from Multimodal