AgentRoboCritic: action-level benchmark reveals how frontier agents really control robots

XiongChenyan · x · 2026-11-21

A CMU/UIUC/Bosch paper, AgentRoboCritic, systematically analyzes how frontier multimodal agents like GPT-6 Astra control robots via low-level actions.

Problem: existing benchmarks report only episode-level success rates, hiding the action-level feedback key to improving embodied agents.

Method: the policy agent states a local intent alongside each action; a critic evaluates the action's outcome and failure mode against that intent, making critique fine-grained and well-posed.

Findings across 12 manipulation benchmarks

The team validated the action-level critics on a human-labeled benchmark.

Original post →

More from Embodied

Embodied channel →