Video models reveal a “Physics Emergence Zone” where motion direction becomes readable
mathemagic1an · x · 2026-07-29
A new interpretability study examines how large video transformers represent physics internally, using probing, subspace geometry, patch-level decoding, and attention ablations.
- The paper identifies a sharp intermediate-depth “Physics Emergence Zone” where physical variables become accessible.
- Speed and acceleration appear from early layers, while motion direction only becomes accessible in the emergence zone.
- Motion direction is encoded in a high-dimensional circular geometry, not as a clean factored variable.
- The authors conclude that video models do not store physics like a classical engine; instead they use distributed representations that are still sufficient for accurate prediction.
Related event: 4M-Parameter Pixel Transformer Emerges as Physical World Model(10 posts)→
More from Research
- SimpleMark: multi-bit LLM watermark bounds per-token distortion with f-divergence budget — rishabh16_ · 2026-10-16
- Stanford paper: generative representation learning cuts tokenizer training compute by 7-8x — HirokatuKataoka · 2026-10-16
- Cohere Labs to present confidence estimation for reasoning models at EMNLP 2026 — Cohere_Labs · 2026-10-16
- Vals RSI Index extrapolates Google models hitting 130% autonomous AI R&D by Oct 2027 — HaydnBelfield · 2026-10-16
- SpatialClaw Pulled Into Agentic Vision Data; Filtering and Evals Were the Real Lift — CMHungSteven · 2026-10-16
- Visual Lean: drag-and-drop GUI for the Natural Numbers Game runs Lean 4 in WASM — MilesCranmer · 2026-10-16