Principia benchmark: top video models score ~0.8 on VBench but under 0.42 on physical consistency
anand_bhattad · x · 2026-09-05
The Principia paper introduces a benchmark testing whether video models obey Newtonian physics via relational invariants between paired objects, sidestepping frame-rate, scale, and camera-calibration ambiguity. It spans eight phenomena—gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass-spring oscillation—using real-world scenes under controlled protocols.
Key findings:
- Across thousands of generations from six state-of-the-art video generators, all score 0.8 on VBench yet none exceeds 0.42 on physical consistency.
- The authors propose a calibration-independent consistency score that quantifies physical violations directly in image space.
- VLMs asked to detect relational physics violations top out at 67% accuracy, with most near chance level.
Related event: Principia Benchmark Exposes Video Models' Poor Physics Understanding(4 posts)→
More from Multimodal
- Kling AI officially launches Kling 4.0: 15 references per task, 30s clips, 10 keyframes — testingcatalog · 2026-10-23
- Kling 4.0 hands-on: multi-shot cyberpunk robot demo shows strong character consistency — testingcatalog · 2026-10-23
- Run Lola Run director Tom Tykwer uses Seedance in hybrid AI filmmaking experiment — Uncanny_Harry · 2026-10-23
- "Still Worth Something" S01E01: a Pixar-style fully AI-animated comedy short — LPGroves · 2026-10-23
- Music publisher Reservoir sues Suno and Anthropic over AI music training — TinfoilTricorn · 2026-10-23
- Halloween AI short: "Careful who you bring home" — glagolef · 2026-10-23