Artificial Analysis launches AA-Robotics benchmark: GPT-6 Astra tops robot arm control at $16.36 per task
ArtificialAnlys · x · 2026-11-04
Artificial Analysis has launched AA-Robotics v0.1, a research preview benchmark where frontier language models in coding agent harnesses directly control a 6-axis robot arm (5 joints + gripper) on simple objectives. Results for the first eight tasks are out.
Key findings:
- GPT-6 Astra ranks first with a score of 69, followed by GPT-6.1 Sol, Gemini 4 Argon, Claude Opus 5.5 and Claude Fable 5.1
- GPT-6 Astra averaged 30 minutes and $16.36 per task
- Performance remains nearly two orders of magnitude worse than human baselines — far too slow and expensive for production — though the team expects this to change quickly
- Post-completion behavior varies sharply: only GPT-6.1 Sol and Gemini 4 Argon stopped within a minute of finishing; GPT-6 Astra submitted 3 minutes late, Claude Fable 5.1 kept going 7 minutes, and Claude Opus 5.5 ran a full 22 minutes past the passing state
The team argues robotics will likely involve strong generalist agents, making it important to track LLM progress on physical tasks.
More from Embodied
- Fruit fly's 166,000-neuron circuit map is being used to drive robot behavior — CurieuxExplorer · 2026-11-04
- Six AgiBot G2 humanoids run a Shanghai convenience store for 24 hours, logging 1,730 tasks — yogthos · 2026-11-04
- Corebrum OS: open-source Physical AI OS now installable on NVIDIA Jetson and any Ubuntu machine — chrismatthieu · 2026-11-04
- Knowing when to stop is hard: Claude Opus 5.5 pushed a crocodile for 22 extra minutes — ArtificialAnlys · 2026-11-04
- Waymo expands South Bay coverage with Berryessa and Milpitas BART connections — reed · 2026-11-04
- Diamandis: a household robot saving 10 hours a week — 520 hours a year — is the robot economy people care about — PeterDiamandis · 2026-11-04