EFE benchmark from 8 universities tests coding agents on 1,156 robot control tasks

机器之心 · wechat · 2026-11-22

Teams from Northwestern, CMU and 6 other universities release EmbodiedFirstExam (EFE), a unified benchmark evaluating coding agents controlling simulated robots across 1,156 tasks, 35 task suites and 47 robots.

Design

Findings on the 456-task core set

Failure attribution: task graphs localize failures to stages; 25% of tests concentrate >80% of failures in one stage. On the double-catch task all 10 agents fail—ClaudeOpus5.5 estimated the catch window at 1.3–1.6s vs 1.4s to reposition the cup.

Limitations: single run per test; annotator GPT-6.1Sol is itself a tested agent from the same family as GPT-6Astra.

Original post →

More from Embodied

Embodied channel →