HandsOn dexterous-manipulation benchmark: best policy 29.1%, GPT-6 Astra just 5.2%
stingning · x · 2026-10-27
stingning shares failure-mode analysis from the HandsOn dexterous manipulation benchmark, based on 800-episode trained-policy rollouts:
- Premature activation: 443 of 800 episodes in Move and Click Mouse terminated on early clicks, often while the hand was still acquiring the mouse.
- Coordination: in Set Dual Knobs, both knobs reached targets simultaneously in only 38/800 episodes; just 5 met the full stabilize-and-release requirement.
- Goal retention: in Open Laptop, policies reached the requested angle and kept pushing past it.
Quantified results: the best multi-task checkpoint (OpenWAM) hits 29.1% task-macro success on Core-50; by hand skill, grasp–transport–release scores 34.9% vs 2.9% for tool use and 2.2% for precision fitting; Transfer-16 success ranges 0.0–0.9%. Zero-finetuned GPT-6 Astra via the moveeef interface managed 26/500 (5.2%) on Core-50 and 1.9% on Transfer-16 across 660 trials. The takeaway: controlling transitions between grasping, operating, stabilizing, and releasing is the core bottleneck.
Related event: GPT-6 Astra Scores Only 5.2% on Zero-Shot Dexterous Manipulation Benchmark(2 posts)→
More from Embodied
- UniAction: unified action conditioning lets one world model handle mixed action types — ZeYanjie · 2026-10-27
- StackChan robot performs an AI-generated skit, orchestrated by Hermes Agent via JSON script — alexcovo_eth · 2026-10-27
- DONIC hits 86.79% on bimanual dexterous tracking, crushing prior SOTA's 9.43% — liuziwei7 · 2026-10-27
- Bedrock Robotics raises $600M as its fully autonomous excavators do paid work on live job sites — Sethwinterroth · 2026-10-27
- AW-0 debuts: one generative world action model plans in code, predicts, and controls robots — trevordarrell · 2026-10-27
- Skyfall AI's SKY-001 world model turns sketches into CAD builds in 3.9s, vs 199s for GPT-6 Astra — rohanpaul_ai · 2026-10-27