Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies
Galbot Team, Xuchuan Chen, Xiaoqian Cheng, Yu Deng, Lihe Ding, Shaocong Dong, Xiangjun Gao, Haozhe Jia, Zekai Li, Zhoujian Li, Yunrui Lian, Sikai Liang, Chenghuai Lin, Dairu Liu, Jiahang Liu, Qingtao Liu, Yuxuan Ma, Zekun Qi, Jiayi Su, He Wang, Ruochen Xu, Tianyu Xu, Xudong Xu, Zhe Xu, Mi Yan, Siming Yan, Li Yi, Ruixi Yu, Jinlu Zhang, Yintianrun Zhang, Zhikai Zhang, Zhizheng Zhang, Yixin Zheng, Weiyi Zhu
cs.RO
2026-09-30
GPT-6 Astra tested as an embodied policy in six robot domains: 92% on RxR navigation, 48% hybrid control on RoboDojo, 0/5 locomotion attempts, 39.86s per call.
Reasoning models have mostly played the planner in robot stacks: write the plan, pick a skill, let a trained policy handle the motion. GPT-6 Astra breaks that division of labor, because it emits numerical robot actions directly, from end-effector targets to navigation moves. The Galbot team asks the practical question: given a model that can output actions, how far is it from a general-purpose embodied policy, where does it break, and what does each decision cost. They evaluate six domains (gripper, dexterous, and mobile manipulation, navigation, locomotion, humanoid loco-manipulation) under each benchmark's native metrics.
Two control paths:
Both run the same loop: observe, generate or review, execute, observe again. Paired comparisons share physical initial states and seeds, and fixed-configuration results are reported separately from development trials where prompts and interfaces were tuned. That is a cleaner protocol than most robot evaluations.
The load-bearing design in Hybrid is restraint. Astra does not take over the trajectory; it steps in when policy proposals stop matching the scene. It modifies 14.4% of executed control steps on RoboDojo and 11.98% on the dexterous benchmark. The policy supplies motion; Astra fixes targets, contact conditions, and task-progress reading.
| Domain | Path | Result | Baseline |
| RoboDojo gripper manipulation (50 instances) | Hybrid | 48% success, Score 62.6 | Direct 26%; π0.5 alone 15.7% |
| RoboLab zero-shot single arm (50 trials) | Direct | 98% | Hybrid 92%; π0.5 36% |
| Dexterous, 10 tasks (0-100 Score) | Hybrid | 61.6 | π0.5 44.2; Direct 16.6 |
| In-hand rotation, at-goal rate | Direct | 0.51% cylinder / 4.40% cuboid | task-specific RL 76.90% / 63.50% |
| RoboCasa365 mobile manipulation (75 episodes) | Hybrid | 38.7% | Direct 33.3%; π0.5 22.7% |
| RxR instruction following, SR | Direct | 92% | LightNav-0 72% |
| HumanoidBench, mean return over 30 tasks | Hybrid | 581.9 | TD-MPC2 338.2 |
Navigation is the strongest domain. With a single forward RGB image and no map, pose, or depth, Astra reaches 92% SR on RxR and 82% on HM3D object search, ahead of five released policies rerun locally on every subset with reported results, on both SR and SPL, with SR advantages over LightNav-0 between 16 and 36 points. The weakness sits in the same table: MP3D object search SPL is 22.43%, so targets get found after heavy detours.
Locomotion is the weakest domain. Five sequential attempts on one obstacle course, none reaching the goal; the fifth stays upright for 30 s and travels 3.8 m, still 6.5 m short, while PASSAGE finishes the course in 13.18 s.
Costs get their own accounting. Fifty RoboDojo instances burn 624.8M tokens under Hybrid and 1.132B under Direct; a 30-second locomotion run needs 250 model calls averaging 39.86 s each, with physics paused during inference.
This evaluation turns "LLM as robot policy" from demo clips into citable numbers. Three takeaways. Zero-shot single-arm semantic pick-and-place is already handled by Direct at 98%, no policy needed. Hybrid fits the "have a policy, but it is flaky" case: touching only 12-14% of steps lifts RoboDojo success from 15.7% to 48%. Navigation beats every locally rerun baseline on a single RGB stream. The cost ledger is equally consequential: tens of seconds per decision rules out real-time control on hardware, and pausing physics only exists in simulation. For humanoids, frozen whole-body controllers plus Astra-selected commands exceed published RL methods on 13 of 30 HumanoidBench tasks, a line worth following.
The authors' own conclusion is a gap between useful task decisions and reliable physical control, most visible in in-hand contact reorganization, locomotion, and completion checks; a trial with the egg still on its side is declared a success in their own text. Policy assistance is not risk-free: on RoboCasa, CoffeeSetupMug succeeds only with the standalone π0.5 while both Astra conditions fail all five seeds, and guidance refined through experience can pass once and fail on the repeat.
Several comparisons deserve discounts. On HumanoidBench, Astra receives privileged state, task geometry, and written guidance, so the 581.9-versus-338.2 margin over published RL methods is not an equal-information comparison. The five locomotion attempts form one adaptation trajectory, not independent trials. The DexJoCo 50% comes from experience-guided trials with guidance rewritten between pairs, not a fixed configuration. Most cells hold five samples, so task-level numbers will move. Navigation baselines were rerun locally under Astra's camera specification, a protocol difference the paper itself flags.