Apple-PI benchmarks video-model reasoning against explicit physical laws

_akhaliq · x · 2026-07-25

Apple-PI introduces a physics-grounded benchmark for video model reasoning

Apple-PI evaluates video models against explicit physical laws instead of only judging whether the final clip looks plausible.

The benchmark uses a three-stage protocol: Perception → Formulation → Deduction.
It applies chain-of-frames prompting, treating the generated video as a visible reasoning trace.
The dataset contains 400 videos spanning 10 classical mechanics tasks.
The authors combine MLLM-based subjective metrics with physics-law objective metrics to diagnose where reasoning fails.
The best video model in the benchmark scores only 0.473, suggesting current systems still fall short on law-grounded physical reasoning.

More from Multimodal