Embodied First Exam: Top Model Scores Just 33% Across 1,090 Robotics Tasks for Coding Agents
ManlingLi_ · x · 2026-10-20
Researchers including Manling Li and Guanya Shi introduce Embodied First Exam, a benchmark probing how far coding agents can go toward general-purpose robots.
- One line of code runs coding agents across 1,090 embodied tasks spanning 31 simulation suites and 44 robots.
- Tasks go well beyond pick-and-place: spatial puzzles, multi-robot Olympic sports, real-time evaluation where the world won't pause, dynamic agile whole-body control, and sim2sim AutoResearch.
- The top-1 model currently completes only 33% of tasks, highlighting a large gap to general embodied capability.
Hosted on Harbor with support for custom harnesses and models, the benchmark aims to become a shared testbed for agentic robotics.
More from Embodied
- Robotics take: IK guarantees no precision edge, hardware—not the solver—is the bottleneck — ChongZzZhang · 2026-10-20
- MPC vs RL for robot tracking: the precision bottleneck is hardware, not the solver — ChongZzZhang · 2026-10-20
- The most important AI device of the next decade won't be a smartphone — VraserX · 2026-10-20
- NUS MAGIC Lab is hiring a postdoc, 2 PhD students and 3 research assistants for robotics — DJiafei · 2026-10-20
- AlphaNIST two-finger gripper screws nuts multiple ways, "it's just PPO" — _Stocko_ · 2026-10-20
- New RL paper: C-Slerp learns 5-cube stacking from scratch with no rewards or pretraining — ben_eysenbach · 2026-10-20