5,400 real-robot rollouts benchmark Claude Fable 5.2 at 65%, ahead of Gemini and GPT
ricklamers · x · 2026-11-14
Poke & Wiggle's "Reality Check Agentic" benchmark measures frontier models as robot agents on real hardware across ten tasks.
- Setup: each agent gets a skill library, motion planner, simulator and 10 demos; it writes skills, simulates 30x, then runs 10 real trials
- 6 frontier models, 5,400 real-world rollouts, spatial generalization measured
- Partial results: Claude Fable 5.2 65%, Claude Opus 5.5 39%, Gemini 4 Argon 35%, GPT-6 Astra 34%
- Authors report never-before-seen success rates and new behaviors; two mystery models pending
More from Embodied
- YC-backed Vexo unveils always-listening AI bracelet that turns conversations into actions — ycombinator · 2026-11-14
- Single-camera capture to animated 3D with gaussian splats and impact frames — andrew_n_carr · 2026-11-14
- Stanford's autonomous robotics textbook and course materials now free online — StanfordHAI · 2026-11-14
- Russ Tedrake's CoRL talk: coupling deployment data with large-scale pretraining — RussTedrake · 2026-11-14
- X-Era AI Lab's world-action model hits 85-88% robot manipulation success rates — jiqizhixin · 2026-11-14
- Figure AI seen most exposed to Nvidia fallout: $3.5-6B compute deal in limbo — kscottz · 2026-11-14