NTU's CodeActionBench tests whether vision-and-code agents can close the robot loop
jiqizhixin · x · 2026-10-23
Nanyang Technological University and collaborators released CodeActionBench (paper + project page), a benchmark evaluating agentic code-as-policy for embodied manipulation.
Motivation
- Existing robot benchmarks either assume a fixed policy interface or skip the full agentic loop: observe, estimate, code, correct, judge completion
- Unlike VLA models that learn action policies from large robot datasets, Code-as-Policy has multimodal models write programs composing robot APIs at inference time
What's evaluated
- Agents must autonomously select observation viewpoints and visual targets, build 3D estimates, construct manipulation poses, write and debug policy code, and self-judge completion
- Depth sensing, external perception/grasp specialists, and privileged scene state are excluded from the interface
- Evaluation jointly checks task results, intermediate progress, and completion judgment, tested on dual-arm scenarios
More from Embodied
- Language as a Sensor: CoRL 2026 Paper Fuses Speech With Vision for Robot Mapping — andreea7b · 2026-10-23
- AI Is Making Hardware Learning Easier: Interactive 3D Learn Arduino App Gets Major Update — burhop · 2026-10-23
- Unitree founder now expects humanoid robots to nail 80% of tasks by 2028, crediting GPT-6 — rohanpaul_ai · 2026-10-23
- FSD 14 lite driver review: near-perfect performance, HW3 is all you need — haydendevs · 2026-10-23
- First real-world sighting of a Tesla Cybercab on public roads — MatthewChang · 2026-10-23
- ETH Zurich's ACP teaches Unitree G1 whole-body ladder climbing with 13 minutes of demos — CyberRobooo · 2026-10-23