Agent-native robotics eval infra: 1,090 tasks, 2,128 variants, 44 robots
ManlingLi_ · x · 2026-10-20
The team explains the motivation behind Embodied First Exam: an agent-native infra evaluating coding-agent skills that matter for robots — reasoning and planning, contact-rich manipulation, dynamics and latency, safety, whole-body athletic control, competition/collaboration, and sim2sim.
- Scale: 1,090 tasks, 2,128 variants, 31 suites, 44 robots, 8 robot types
- 382 brand-new tasks in 8 original suites (RoboARC, RoboOlympics, RoboPaint, DexHandExam, RoboDynamicExam, RoboEngineer, SafetyExam, etc.)
- Plus hard tasks drawn from 23 community benchmarks like RoboDojo, BEHAVIOR-1K, RoboCasa, LIBERO, MyoSuite
- Two modes: Standard approximates the real world (camera-only, partial observation, no resets, real time); Privileged grants full simulator access as an upper bound
More from Embodied
- Robotics take: IK guarantees no precision edge, hardware—not the solver—is the bottleneck — ChongZzZhang · 2026-10-20
- MPC vs RL for robot tracking: the precision bottleneck is hardware, not the solver — ChongZzZhang · 2026-10-20
- The most important AI device of the next decade won't be a smartphone — VraserX · 2026-10-20
- NUS MAGIC Lab is hiring a postdoc, 2 PhD students and 3 research assistants for robotics — DJiafei · 2026-10-20
- AlphaNIST two-finger gripper screws nuts multiple ways, "it's just PPO" — _Stocko_ · 2026-10-20
- New RL paper: C-Slerp learns 5-cube stacking from scratch with no rewards or pretraining — ben_eysenbach · 2026-10-20