Frozen GPT-6-Astra used directly as robot policy hits 92.1% on RoboMME, beating human baseline
hankyang94 · x · 2026-10-17
A new arXiv paper from Heng Yang and coauthors asks whether a frozen frontier VLM can serve as a robot policy for memory-dependent manipulation.
- Minimal approach: GPT-6-Astra is used directly as the policy — the full sampled visual history of the episode stays in context, and the model outputs every end-effector waypoint and gripper command, with no RoboMME-specific training.
- Results: On RoboMME's 16 memory-dependent tasks (temporal, spatial, object, procedural) across 800 test episodes, it scores 92.1% — above the previously reported human baseline of 90.5% (achieved with an oracle skill interface), far ahead of the best leaderboard entry (60.8%) and a hierarchical system pairing the same model with a trained policy (79.1%). It matches or exceeds human performance on 13 of 16 tasks.
- Failures: 63 failures concentrate in two tasks, mostly premature completions and mistimed or unintended button presses.
- Implication: the results blur the usual division between long-horizon planning, memory, and end-effector control, raising the question of how much of that stack a single general-purpose model can take over. Code, prompts, and all episode logs are public.
Related event: GPT-6-Astra as Robot Policy Tops RoboMME, Beating Human Baseline(2 posts)→
More from Embodied
- Contact-UAN: MIT's residual world model fixes humanoid sim-to-real with 6.5 minutes of data — kevin_zakka · 2026-10-17
- ROS 2 LiDAR localization runs 390 m outdoors on handheld Livox MID-360 — rsasaki0109 · 2026-10-17
- Embodied Astra tops RoboMME at 92.1% — beating humans by just keeping full visual history in context — hankyang94 · 2026-10-17
- 100% blind developer built AccessSlice with Claude to make 3D printing accessible — Mrblindguardian · 2026-10-17
- ROS weekly: ROSCon China kicks off in Shanghai, ROS By-The-Bay set for Oct 22 at Intrinsic — kscottz · 2026-10-17
- Simulated robotics eval: Claude Opus 5.5 preferred over Opus 5 in 14 of 16 comparisons — qinzytech · 2026-10-17