Apple-PI benchmarks 11 video models on physics-grounded reasoning, and the best scores 0.473
_akhaliq · x · 2026-07-24
What the paper proposes
Apple-PI is a new benchmark for evaluating video models through the lens of physical law, instead of judging only whether the final output looks plausible.
Core components
- Orchard: a dataset of 400 videos across 10 classical mechanics tasks.
- Benchmark protocol: a three-stage reasoning flow — Perception, Formulation, Deduction — using chain-of-frames prompting on annotated first frames.
- Evaluation suite: combines subjective MLLM scoring with physics-law-grounded objective measures.
Main finding
Benchmarking 11 models shows current video models are still far from reliable law-grounded world simulators. The best model scores only 0.473. The authors argue failures cluster around a Perception → Formulation → Deduction bottleneck, weak multi-law transfer, and a persistent sim-to-real gap.
Related event: Apple-π Benchmark Reveals Shortcomings in Video Models' Physical Reasoning(5 posts)→
More from Multimodal
- CapCut plugin is coming to ChatGPT for AI-powered video editing workflows — HeyZoyaKhan · 2026-10-27
- Magnific teases October 28 release: 'We're making it move' — cuenca · 2026-10-27
- Midjourney --sref demo: retro Americana Batman art series — gizakdag · 2026-10-27
- AI video is a new medium, not cinema or animation — and it's perfect for depicting dreams — Kyrannio · 2026-10-27
- Dad turns his toddler's 31-year-old quote into a song via SUNO kids' choir — NathanpmYoung · 2026-10-27
- Band's local-AI music video hits 200k views: 900 generations kept 64, 3-second clips beat face drift — Hwang0500 · 2026-10-27