BUPT study: RoPE attention decay causes video diffusion models to violate physics
BUPT-CIST · hf · 2026-09-22
BUPT researchers present the first interpretability study of the "motion planning" process in text-to-video diffusion models. They find trajectories form in early denoising stages ("first shape, then details"), identify attention heads driving motion planning, and show Rotary Position Embedding (RoPE) induces excessive spatial attention decay that prematurely locks candidate regions into physically implausible positions. A lightweight fix—scaling RoPE frequencies across denoising steps—reduces the decay, and both training-free and training-based experiments confirm improved physical commonsense in generated videos.
Related event: Study Explains Why Video Diffusion Models Break Physics(2 posts)→
More from Multimodal
- Endless screensaver of self-assembling quantum cryostats, made with Gemini 4 Carbon — hapticdata · 2026-11-12
- ADM 3 Cloud flops in image evals: ranks #86 of 107 in text-to-image — ArtificialAnlys · 2026-11-12
- ADM 3 Cloud ranks #47 of 80 at photo enhancement and restoration — ArtificialAnlys · 2026-11-12
- Apple's ADM 3 Cloud tested: #42 at image editing, #85 at text-to-image — ArtificialAnlys · 2026-11-12
- Object-level edits are ADM 3 Cloud's strongest skill, ranking #27 of 80 — ArtificialAnlys · 2026-11-12
- Apple's ADM 3 Cloud image model ranks #42 in editing and #85 in text-to-image on Artificial Analysis arena — ArtificialAnlys · 2026-11-12