AgentRoboCritic: action-level benchmark reveals how frontier agents really control robots
XiongChenyan · x · 2026-11-21
A CMU/UIUC/Bosch paper, AgentRoboCritic, systematically analyzes how frontier multimodal agents like GPT-6 Astra control robots via low-level actions.
Problem: existing benchmarks report only episode-level success rates, hiding the action-level feedback key to improving embodied agents.
Method: the policy agent states a local intent alongside each action; a critic evaluates the action's outcome and failure mode against that intent, making critique fine-grained and well-posed.
Findings across 12 manipulation benchmarks
- Frontier agents show closed-loop recovery from local execution failures, reliable semantic grounding across scenes, and strong zero-shot manipulation without task-specific adaptation
- Bottlenecks are physical execution steps like grasping and alignment
- Recovery is uneven across failure modes
The team validated the action-level critics on a human-labeled benchmark.
More from Embodied
- Reverse-engineering Apple Watch: a single Claude Code prompt builds a physics watch face in 30 minutes — mtizard · 2026-11-21
- Robotics Founder: Housekeeping Bots Should Target Mundane Upkeep, Not Party Tricks — Ben_Reinhardt · 2026-11-21
- GPT-6.2 Astra tops AA-Robotics v0.1 with 75% fewer arm commands than Claude — ArtificialAnlys · 2026-11-21
- Heave-Ho: decentralized framework coordinates teams of 2–10 humanoids on long-horizon tasks — yuewang314 · 2026-11-21
- USC's Heave-Ho coordinates humanoid robot teams with zero network messages, 86% success at 120x faster planning — yuewang314 · 2026-11-21
- Unitree G1 humanoid robot 'gets a job' at U-Haul in real-world demo — yuewang314 · 2026-11-21