Top agent solves only 33% of Embodied First Exam; GPT-6 Astra took 40 min per bolt
ManlingLi_ · x · 2026-10-20
First leaderboard results from Embodied First Exam: the strongest agent, GPT-6 Astra, scores just 33% on the core leaderboard.
- Privileged ≠ Standard: tasks agents solve with full simulator access mostly fail under real-robot rules (camera-only, no resets). Astra solves only about a third of Standard and Standard-realtime tests; Claude Opus 5.5 and Gemini 4 Argon each solve dozens of tests Astra fails. No single agent covers it all
- Where agents break: dexterity and tool use, dynamic control and balance, long-horizon chores. Even with privileged access, Astra took 40 minutes to drive one bolt on an IKEA Stegesund and ran out of time on the second
- Where they shine: tasks where motion can be planned in advance, resets allow free retries, or rules are explicit
Leaderboard and full report linked in the post.
More from Embodied
- MoCaP: zero-shot mobile manipulation for robots from passive human videos — sherryyangML · 2026-10-20
- Lucky Engine 2026.3 launches: photo-to-simulation for robot policy training — chris_j_paxton · 2026-10-20
- Berkeley EMBER humanoid seminar series: CMU's Guanya Shi on generalist humanoids — GuanyaShi · 2026-10-20
- Apple reportedly set to unveil touchscreen, eSIM MacBook on Oct 28 — xiaohu · 2026-10-20
- Same interface drives a real robot: buzz-wire demo and EmbodiedSafetyExam — ManlingLi_ · 2026-10-20
- RoboEngineer demo: AutoResearch on sim2sim — ManlingLi_ · 2026-10-20