EgoSteer trains a dual-dexterous-hand system on 9.6K hours of human hand video

机器之心 · wechat · 2026-07-23

EgoSteer uses 9.6K hours of human hand video to train a dual-dexterity manipulation system

Lingchu Intelligence and the PKU–Lingchu joint lab released EgoSteer, an open-source dual-dexterous-hand manipulation model and full-stack system.

Data pipeline: EgoSmith turns noisy first-person hand videos into training data through filtering, 4D motion estimation, multi-level language labeling, and multi-stage quality control.
Scale: the resulting dataset spans 9.6K hours, 2.09 million clips, and 1.04 billion frames across 12 open datasets.
Model design: a Qwen3-VL backbone is paired with a diffusion/flow-matching action expert and a lightweight world-model expert used only during training.
System: the robot stack supports teleoperation, deployment, and seamless human takeover via relative action mapping.
Results: EgoSteer achieved 75% average success across 40 tasks, handled unseen semantic tasks with about 62% success, and learned long-horizon tasks from as few as 120–200 demonstrations.
Open source: code and weights are already open; data is being opened soon.

More from Embodied