HandsOn Benchmark Released and Open-Sourced for Generalist Dexterous Manipulation

To measure progress of general dexterous manipulation models like Natura-1, @stingning's team released and open-sourced the simulation platform and benchmark HandsOn on October 27. Results show even today's strongest policies remain far from general dexterous manipulation.

Confirmed

• The benchmark includes 66 human-verified tasks (50 core + 16 transfer) and 18,000 curated human demonstrations, roughly 56 hours of data — all collected in simulation with three synchronized views, augmented by re-rendering and prompt randomization; the dataset is split into 15,178 training and 2,822 validation demonstrations
• Evaluation has two parts: Core-50 provides demonstrations for 50 tasks for policy learning and evaluates execution; Transfer-16 contains 16 transfer tasks with no target-task demonstrations — 7 asset/relation tasks, 6 skill-composition tasks and 3 others — testing zero-demonstration transfer
• On Core-50, the best multi-task checkpoint, OpenWAM, reached only a 29.1% macro-average task success rate, with wide capability gaps: pooled across 8 training checkpoints, pick-carry-place tasks hit 34.9% success, while tool-use-on-another-object tasks were as low as 2.9%
• Fully open-sourced resources include task environments, success detectors, evaluation tools, and the PhysX demonstration dataset

Why it matters

• With the emergence of general dexterous manipulation models like Natura-1, the field lacked a unified benchmark for measuring progress toward generalizable dexterous manipulation — HandsOn fills that gap
• The 2.9% success rate on tool-use tasks shows dexterous manipulation, especially tool interaction, remains a glaring weakness in robot learning, giving future research a clear direction and a public, reproducible evaluation foundation

2026-10-27 ~ 2026-10-27 · 5 related posts

Full story(2 episodes)→

• Episode 1: HandsOn Benchmark Released and Open-Sourced for Generalist Dexterous Manipulation(2026-10-27, 5 posts)
• Episode 2: GPT-6 Astra Scores Only 5.2% on Zero-Shot Dexterous Manipulation Benchmark(2026-10-27, 2 posts)

• [source] HandsOn benchmark: 66 verified tasks and 18,000 human demos for dexterous manipulation — stingning · 2026-10-27
• HandsOn splits evaluation: Core-50 execution vs Transfer-16 zero-demo transfer — stingning · 2026-10-27
• [source] Best policy hits only 29.1% on HandsOn; tool use drops to 2.9%, transfer near zero — stingning · 2026-10-27
• [source] HandsOn platform open-sourced: 18K human demos, tasks, eval tools for dexterous policies — stingning · 2026-10-27

1 near-duplicate retellings: stingning