GPT-6 Astra robot evals show strong task decisions but weak physical control
Galbot · hf · 2026-10-01
Galbot systematically evaluated GPT-6 Astra as a general-purpose embodied policy across six domains:
- Gripper manipulation: corrects task targets and preps contact conditions; hybrid control with π0.5 hits 48% success on a RoboDojo subset.
- Dexterous manipulation: 50% success across ten DexJoCo trials with hybrid control, but direct in-hand control struggles to coordinate finger contacts.
- Mobile manipulation: 38.7% success on RoboCasa365.
- Navigation: leads local comparisons with 92% on RxR instruction following and 82% on HM3D object search, though search involves substantial detours.
- Locomotion: unreliable — none of five sequential obstacle-course attempts reached the goal.
- Humanoid loco-manipulation: beats baselines on 13 of 30 HumanoidBench tasks.
The core finding: a gap between useful task decisions and reliable physical control. Inference latency is a major constraint — policy-assisted and direct control consumed 624.8M and 1.132B tokens respectively across conditions, and a 30-second locomotion run required 250 model calls averaging 39.86 seconds each, with physics paused during inference.
More from Embodied
- YC-backed Ornadyne's flapping-wing drone was classified as a bird by Air Force detectors — import_jmr · 2026-11-16
- Uber starts offering Amazon's Zoox robotaxi rides in Las Vegas — Polymarket · 2026-11-16
- Ant Lingbo Embodied AI Challenge crowns TARS team after 48-hour real-robot hackathon — 新智元 · 2026-11-16
- Tesla FSD fleet tops 7 billion city miles, adding ~500M in under four weeks — XFreeze · 2026-11-16
- Ropedia Releases Xperience 2: 1,500+ Scanned Real-World Scenes Paired With Egocentric Demos — liuziwei7 · 2026-11-16
- Hangzhou's Manycore Tech launches Aholo MS07 3D laser scanner with CHCNAV — teortaxesTex · 2026-11-16