GPT-6 Astra tops AA-Robotics v0.1, but frontier LLMs remain ~100x slower than humans at robot arm tasks

ArtificialAnlys · x · 2026-10-15

Artificial Analysis launched AA-Robotics v0.1, a research preview that gives frontier LLMs control of a 6-axis robot arm (5 joints + gripper) to test understanding, planning, and acting in the physical world, with results for the first eight tasks.

Setup: Models run in isolated containers with only brief instructions (an HTTP API, the arm model — a Hiwonder NexArm — and how to submit). They see the scene via four cameras (three around the table, one on the wrist), write and run code to plan movements, and are graded zero-shot against pass criteria (100 for a pass, partial credit otherwise).

Leaderboard: GPT-6 Astra scores 69, ahead of GPT-6.1 Sol (65), Gemini 4 Argon (55), Claude Opus 5.5 (49) and Claude Fable 5.1 (46).

Key findings:

• Models average 30 minutes per tabletop task
• Costs vary widely: GPT-6.1 Sol is cheapest at $1.93/task; GPT-6 Astra costs $16.36, over 8x more for 4 extra points
• Best models are 100x slower and 10x more expensive than humans
• Three common failure modes: missed grasps/placements, poor spatial reasoning (knocking over their own towers), and misjudging success — often continuing past a passing state

The authors conclude LLMs in coding agent harnesses have only recently become capable of directly manipulating robot arms for simple tasks — far from production-ready, but improving quickly.

Original post →

More from Embodied

• Why is my Oura ring talking to me like a therapist? — var_epsilon · 2026-10-15
• SONIC now runs on full-size humanoids: Agibot A3 and UBTech Walker C1 — zhengyiluo · 2026-10-15
• New RealSense D555/D585 pack quad-core SoCs with built-in AI models — chrismatthieu · 2026-10-15
• Feather Robotics ships robots back-to-back to New Zealand and India — darian314 · 2026-10-15
• RealSense D415 drops under $300 with ~750 units in stock at Digikey — chrismatthieu · 2026-10-15
• ASENA: a coding agent writes robot programs on the fly, no training needed — zhengyiluo · 2026-10-15

Embodied channel →