Video-model activations linearly expose position and predict collisions early
mathemagic1an · x · 2026-07-29
The thread argues that video models can encode physical state far more explicitly than they appear to from the outside.
- X/Y coordinates are linearly decodable from activations.
- Deeper layers can anticipate collisions several steps ahead.
- That suggests the model is not just interpolating pixels, but forming higher-level representations of objects and their relationships.
- The author uses this as motivation for the Jacobian-lens experiment that turns those internal directions into controllable motion.
Related event: 4M-Parameter Pixel Transformer Emerges as Physical World Model(10 posts)→
More from Multimodal
- Dev builds MCP feeding Claude 3,000 real launch videos to fix samey AI output — Designer_Landscape25 · 2026-10-16
- Mirelo SFX V2.0 launches on fal with dialogue-preserving video sound effects — gorkem · 2026-10-16
- Moss-Image samples show startlingly lifelike film-style photography — AgeNo5351 · 2026-10-16
- Snap Spectacles + Gemini 4 Argon demo designs LEGO builds from your brick pile — stspanho · 2026-10-16
- Stanford paper: generative representation learning cuts tokenizer training compute by 7-8x — HirokatuKataoka · 2026-10-16
- LTX 2.5 background swap workflow keeps subjects nearly pixel-perfect — Omrbig · 2026-10-16