Embodied First Exam: top agent GPT-6 Astra solves just 32.9% of 456 core robotics tasks
ManlingLi_ · x · 2026-11-09
A multi-university team (Northwestern, CMU, Stanford, UC Berkeley, with advisors including Li Fei-Fei, Pieter Abbeel, Dawn Song, and Eric Xing) launched Embodied First Exam at CoRL Late-Breaking Results — a comprehensive benchmark evaluating coding agents on real robotics tasks.
- Scale: 1,156 tasks, 2,288 tests, 35 suites, 47 robots ranging from bimanual basketball dunks and dexterous hands to mobile manipulators, quadrupeds, humanoids, and musculoskeletal models
- Three modes: Privileged (full simulator state, frozen-start replays), Standard (robot API only, graded on the live run), and Realtime (robot never stops, clock runs)
- Four control styles: direct control, code-to-control, training a policy, and mixed; an offline verifier gives pass/fail
Results: even the strongest agent, GPT-6 Astra, solves only 32.9% of the 456 core tasks (one attempt each); Claude Opus 5.5 scores 23.5%, Gemini 4 Argon 19.5%, DeepSeek V4.1 Flash 2.6%. Code agents have passed many digital-world exams, but their first exam in the physical world remains largely unsolved.
Related event: New Embodied Benchmark Leaves Best Agents at 32.9%(2 posts)→
More from Embodied
- Tesla Optimus engineer builds GROXIE, a countertop robot assistant powered by Grok — Scobleizer · 2026-11-09
- RLT paper wins CoRL oral: sample-efficient real-world RL to fine-tune VLA models — svlevine · 2026-11-09
- Embodied First Exam: 1,090 embodied tasks stump coding agents, best solves just 32.9% — LINJIEFUN · 2026-11-09
- Berkeley's AsyncVLA runs VLA navigation in real time, 40% higher success under 6s latency — svlevine · 2026-11-09
- NVIDIA AI Day Seoul Draws 2,000+ Developers to Showcase Physical AI, Robotics and Agentic AI — NVIDIAAI · 2026-11-09
- NVIDIA powers Korea's physical AI: Samsung sims 1600x faster, LG robots in factories — NVIDIA Blog · 2026-11-09