Apple-PI benchmarks 11 video models on physics-grounded reasoning, and the best scores 0.473
_akhaliq · x · 2026-07-24
What the paper proposes
Apple-PI is a new benchmark for evaluating video models through the lens of physical law, instead of judging only whether the final output looks plausible.
Core components
- Orchard: a dataset of 400 videos across 10 classical mechanics tasks.
- Benchmark protocol: a three-stage reasoning flow — Perception, Formulation, Deduction — using chain-of-frames prompting on annotated first frames.
- Evaluation suite: combines subjective MLLM scoring with physics-law-grounded objective measures.
Main finding
Benchmarking 11 models shows current video models are still far from reliable law-grounded world simulators. The best model scores only 0.473. The authors argue failures cluster around a Perception → Formulation → Deduction bottleneck, weak multi-law transfer, and a persistent sim-to-real gap.
Related event: Apple-π Benchmark Reveals Shortcomings in Video Models' Physical Reasoning(5 posts)→
More from Multimodal
- Midjourney style code share: --sref 2912175708 — tisch_eins · 2026-09-11
- Astra storyboards plus Minimax H3 per-shot generation boost video success rates — Hailuo_AI · 2026-09-11
- MiniMax H3 MAX nails cooking anime clips: 15-second curry demo with prompts shared — Hailuo_AI · 2026-09-11
- MiniMax Music Production Toolkit 2.5 for ComfyUI adds full mastering chain — Vivid_Promise1700 · 2026-09-11
- New Node Finder for ComfyUI ranks fresh nodes by star velocity and recency — Luke2642 · 2026-09-11
- Using a finisher move on one mosquito with MiniMax H3 MAX — the bug survives — Hailuo_AI · 2026-09-11