AA-Robotics debut: GPT-6 Astra tops real robot-arm benchmark but runs ~100x slower than humans
机器之心 · wechat · 2026-10-15
ArtificialAnalysis launched AA-Robotics v0.1, having frontier models control a real 6-axis arm via coding agents on eight zero-shot tasks.
First results: GPT-6 Astra scored 69.4, GPT-6.1 Sol 64.6 (with the most perfect task scores at five), Gemini 4 Argon 55.3, Claude Opus 5.5 / Fable 5.1 48.8 / 46.3. All models passed the easiest tasks; only Astra poured water and wrote "hello world"; nobody aced the block tower — Sol built a valid tower then knocked it over.
Efficiency gap: tasks took 30 minutes on average vs a 20-second human baseline (nearly two orders of magnitude slower); 60% of time goes to planning and only 13% to actual arm motion. Astra cost $16.36 per task vs $1.93 for Sol, which scored just 4.8 points lower. Grasp success rates hovered at 28-45%.
The debate: general models can now "act," but embodied-AI founders argue reliable physical interaction still needs dedicated data, VLA models and control policies. The first batch includes no Chinese models — whether Qwen, Kimi, GLM or DeepSeek make the next list is worth watching.
More from Embodied
- Berkeley's Value Forcing: enforcing Bellman consistency at every noise level for accurate RL value functions — berkeley_ai · 2026-11-04
- 22 Creative SAM 3.1 Use Cases From Meta's AI Developer Hackathon, From Wildfire Detection to Document Reassembly — Vjeux · 2026-11-04
- Ex-OpenAI robotics intern breaks down viral Astra robot-painting demo on Cerebras podcast — AccBalanced · 2026-11-04
- Embodied Agent Arena: Closed-Weight VLMs Lead All Five Robot Task Domains, Open-Weight Models Lag — Haojian Huang · 2026-11-04
- PrimeBot's PrimeSim: a generalist world model simulator trained on cross-embodiment demos — Travis Davies · 2026-11-04
- Physical AI Hackathon in SF Nov 7–8 features Dream-1 world action model and robot arms — Scobleizer · 2026-11-04