WorldOracle: New Benchmark Tests World Model Predictions Against Real Robot Executions
heyshrutimishra · x · 2026-10-22
WorldOracle: A Real-World Benchmark for Action-Conditioned World Models
Researchers released WorldOracle, a benchmark that checks whether the "predicted futures" generated by action-conditioned world models stay coherent with real robot executions: given the robot's first camera frame and its motor commands, does the model's video rollout reproduce the actual physical consequences and task-state transitions?
Key design
- Multiple embodiments and interaction dynamics: AgiBot G2 dual-arm robot, YAMBI bimanual platform, and human-hand egocentric manipulation
- Scoring: Physical State Correctness (PSC), Interaction Spatial Alignment (ISA), Key Interaction Area Fidelity (KIAF), Multi-View Action Coherence (MVAC)
- Includes long, multi-segment episodes with both successful and failed executions to differentiate models
The leaderboard is open on AgiBot G2 · 30s with no published model results yet. Paper, code (GitHub), and dataset (Hugging Face) are available. Motivation: robotics teams want to use world-model videos as training data and to test policies without running real hardware—WorldOracle tests whether that holds up.
More from Embodied
- Asimov 1 robot ships IKEA-style manual: build it in under a week — TinfoilTricorn · 2026-10-22
- Humanoid robot at NVIDIA GTC runs Gemma 4 E2B for multimodal input — patloeber · 2026-10-22
- ARK's Sam Korus: Tesla Optimus academy and David Silver's super learner converge — skorusARK · 2026-10-22
- Nirva raises $33M for a $249 AI pendant that only listens to you — TechCrunch AI · 2026-10-22
- Qualcomm Academy launches edge AI certification with three hands-on projects on Arduino UNO Q — alexcovo_eth · 2026-10-22
- Mori is a cute robot mushroom that doesn't really say or do anything — Wired AI · 2026-10-22