EmbodiedFirstExam: 1,156-task benchmark shows top coding agents solve only half

新智元 · wechat · 2026-11-19

Researchers from Northwestern, CMU and six other universities released EmbodiedFirstExam (EFE), a unified benchmark wiring coding agents into simulated robots: 1,156 tasks, 2,288 tests, 47 robots across 23 simulators, open-sourced with code and trajectories.

Key results

Analysis findings

Limitations: single runs per test, some labels produced by a tested model, mostly simulation-only; the team plans repeats and systematic real-robot evaluation.

Original post →

More from Embodied

Embodied channel →