FULL STORY
AA-Robotics: From Benchmark Launch to Models That Can't Stop
Artificial Analysis debuted AA-Robotics with GPT-6 Astra topping robotic-arm tasks, then added a stop-condition test where most models kept working pointlessly after finishing, exposing poor termination judgment.
2026-10-15 ~ 2026-11-04 · 2 episodes · 5 posts
Episode 1 · Artificial Analysis Launches AA-Robotics Benchmark; GPT-6 Astra Tops Ranking (2026-10-15, 3 posts)
Artificial Analysis released AA-Robotics v0.1, the first benchmark testing LLMs on real robot-arm tasks; GPT-6 Astra ranked first, though even the best model remains roughly 100 times slower than humans.
- GPT-6 Astra tops AA-Robotics v0.1, but frontier LLMs remain ~100x slower than humans at robot arm tasks — ArtificialAnlys · 2026-10-15
- AA-Robotics debut: GPT-6 Astra tops real robot-arm benchmark but runs ~100x slower than humans — 机器之心 · 2026-10-15
- Artificial Analysis launches AA-Robotics benchmark ranking LLMs on robot tasks — ObiWanCanownme · 2026-10-15
Episode 2 · Robot-arm test finds most LLMs don't know when to stop (2026-11-04, 2 posts)
An Artificial Analysis robot-arm test showed Claude Opus 5.5 kept pushing uselessly for 22 minutes after finishing its task, with only GPT-6.1 Sol and Gemini 4 Argon stopping within a minute of completion—showing 'knowing when to stop' remains hard for LLMs.
- Knowing when to stop is hard: Claude Opus 5.5 pushed a crocodile for 22 extra minutes — ArtificialAnlys · 2026-11-04
- Only GPT-6.1 Sol and Gemini 4 Argon stopped within a minute on robot arm task — ArtificialAnlys · 2026-11-04