Open-AoE opens 2,000 hours of egocentric manipulation video for robot learning

inclusionAI · hf · 2026-07-21

Open-AoE releases an open egocentric manipulation dataset and toolchain

Open-AoE is a community-oriented dataset and toolchain for embodied learning that covers the full pipeline from smartphone capture to model training.

• Scale: about 2,000 hours of manipulation video
• Collection: gathered in natural environments by 500+ contributors using 400+ smartphones
• Annotations: text labels, MANO-based hand poses, camera trajectories, and temporally localized atomic actions
• Pipeline: temporal action segmentation, semantic annotation, hand reconstruction, and camera trajectory reconstruction
• Toolchain: visualization, cross-embodiment retargeting, model-specific data conversion, and training recipes for VLA policies, WAMs, and World Models

The project aims to lower the barrier for data contribution and reuse in human-to-robot transfer and world modeling.

Original post →

More from Embodied

• NVIDIA open-sources FLORA, a robotics data engine ~125x cheaper than GPT-6 Astra — nvidia · 2026-10-27
• Embodied AI startups need a different operational discipline, investor says — ___Mufasaa · 2026-10-27
• π0.5 study: raising demos from 10 to 438 lifts Place Ball in Bowl success from 7% to 83% — stingning · 2026-10-27
• GPT-6 Astra controls dexterous hands zero-shot at just 5.2% success in 500 trials — stingning · 2026-10-27
• HandsOn benchmark: 66 verified tasks and 18,000 human demos for dexterous manipulation — stingning · 2026-10-27
• DexPort: a handheld interface with a real robot hand for dexterous-hand data collection — pabbeel · 2026-10-27

Embodied channel →