WorldOracle: New Benchmark Tests World Models Against Real Robot Executions
机器之心 · wechat · 2026-10-16
Researchers from Princeton, UT Austin, U Toronto, NVIDIA and Zhiyuan released WorldOracle: given an initial frame and action sequence from a real operation, a world model generates a predicted video scored against the actual execution — correctly predicting a failure counts as correct.
Evaluating DreamDojo, Cosmos3, IRASim, GE-Sim 2.0 and others revealed two issues: robot arms track well while objects don't (IRASim: arm 78.8 vs object 29.1), and accuracy degrades over long horizons (DreamDojo 50.56→27.08; Cosmos3 53.99→26.60; GE-Sim 2.0 leads at 63.33).
The platform covers Zhiyuan G2 and YAM embodiments plus first-person hand scenarios, with tasks spanning rigid, deformable, fluid and articulated objects, three tracks (Core/Policy/Long), and four metrics (physical state, spatial alignment, interaction-region fidelity, multi-view consistency) that agree with human judgment 88.1–93.2% of the time. Submissions opened Oct 16.
More from Embodied
- Brain implants are restoring speech to people with ALS, Science reports — EricMTrautmann · 2026-10-18
- Meta wearables VP runs his AI agent across 3 phones and 23 pairs of glasses — nikola_mr64990 · 2026-10-18
- AMI-EV: spinning prism gives event cameras stable texture, Science Robotics 2024 — RexDouglass · 2026-10-18
- Neuralink patients control robotic arms, drones and games with thought alone — XFreeze · 2026-10-18
- Indie roboticist shows first neck actuation attempt for humanoid robot, AI-assisted design — _Stocko_ · 2026-10-18
- Retired Penn professor arrested for blocking Uber Eats delivery robot — johnnyApplePRNG · 2026-10-18