Embodied First Exam: 1,090 embodied tasks show agents crumble outside privileged settings
ManlingLi_ · x · 2026-11-18
Manling Li's team released Embodied First Exam (EFE): one line of code runs coding agents across 31 simulation suites, 44 robots, and 1,090 embodied tasks, covering spatial-constraint puzzles, multi-robot Olympic sports, real-time eval where the world won't pause, dynamic whole-body control, and sim2sim AutoResearch. Try it: harbor run -d embodied-first-exam -a <harness> -m <model>.
BiGym 2.0 is one of the 31 suites. EFE's three settings tell a stark story: under privileged access, Fable 5.2 and GPT-6 Astra solve all 10 core tasks; using only the robot's own cameras, the best score drops to 6; in realtime, at most 1 — sharply quantifying the privileged-to-real gap.
More from Embodied
- FUSE San Francisco robotics localization conference sells out, agenda revealed — kscottz · 2026-11-18
- Google launches Gemini 4 Hydrogen robotics plugin: train real robots via conversation — GoogleDeepMind · 2026-11-18
- Agent kept bending arms wrong, so this dev wired MuJoCo into an MCP server — usefulwrong · 2026-11-18
- HRT1 accepted to RA-L: a robot learns mobile manipulation from a single human video — YuXiang_IRVL · 2026-11-18
- Neuralink patents living-neuron brain-computer interface with "layer zero" graft — XFreeze · 2026-11-18
- VITURE launches $599 Phantom Beast XR glasses co-branded with Phantom Blade Zero — Scobleizer · 2026-11-18