Open-AoE opens 2,000 hours of egocentric manipulation video for robot learning

inclusionAI · hf · 2026-07-21

Open-AoE releases an open egocentric manipulation dataset and toolchain

Open-AoE is a community-oriented dataset and toolchain for embodied learning that covers the full pipeline from smartphone capture to model training.

Scale: about 2,000 hours of manipulation video
Collection: gathered in natural environments by 500+ contributors using 400+ smartphones
Annotations: text labels, MANO-based hand poses, camera trajectories, and temporally localized atomic actions
Pipeline: temporal action segmentation, semantic annotation, hand reconstruction, and camera trajectory reconstruction
Toolchain: visualization, cross-embodiment retargeting, model-specific data conversion, and training recipes for VLA policies, WAMs, and World Models

The project aims to lower the barrier for data contribution and reuse in human-to-robot transfer and world modeling.

More from Embodied