WorldOracle benchmark from Princeton, NVIDIA, AgiBot exposes hallucinations in action-driven world models
jiqizhixin · x · 2026-10-24
When a robot hand closes on empty air, action-conditioned world models often generate a video showing a successful grasp — plausible-looking rollouts that diverge from reality.
- What it is: WorldOracle (ArXiv 2610.18826), a benchmark from Princeton, UT Austin, University of Toronto, NVIDIA, and AgiBot that scores action-driven world models against real robot execution recordings used as ground truth. Developers submit videos generated from a given initial frame and action sequence; the platform compares them frame-by-frame with the real trajectory.
- Findings: Seven representative models — including DreamDojo, Cosmos 3, IRASim, and GE-Sim 2.0 — were evaluated, revealing consistent failures where models hallucinate the most common successful ending instead of predicting what actually happens.
Related event: WorldOracle benchmark reveals world models hallucinate robot action success(2 posts)→
More from Embodied
- Baidu's Apollo Go and Lyft's Freenow unveil RT6 robotaxi in London, targeting 2027 driverless launch — Baidu_Inc · 2026-10-24
- $3000 folding iPhone hit by bright screen lines, hinge design fault suspected — LinusEkenstam · 2026-10-24
- Dev rides 1,000km of Chinese high-speed rail, snaps Shenzhen's police patrol robot — LoganGrasby · 2026-10-24
- Niantic Spatial CEO Inhi Suh on sim-to-real infrastructure for embodied AI — ExtensionEcho3 · 2026-10-24
- RoboSimStudio Turns One Phone Video Into an Editable Robot Physics Twin in Under an Hour — lukas_m_ziegler · 2026-10-24
- Waymo robotaxis are now doing In-N-Out drive-thru runs fully on their own — FanaHOVA · 2026-10-24