EgoSteer trains a dual-dexterous-hand system on 9.6K hours of human hand video
机器之心 · wechat · 2026-07-23
EgoSteer uses 9.6K hours of human hand video to train a dual-dexterity manipulation system
Lingchu Intelligence and the PKU–Lingchu joint lab released EgoSteer, an open-source dual-dexterous-hand manipulation model and full-stack system.
- Data pipeline: EgoSmith turns noisy first-person hand videos into training data through filtering, 4D motion estimation, multi-level language labeling, and multi-stage quality control.
- Scale: the resulting dataset spans 9.6K hours, 2.09 million clips, and 1.04 billion frames across 12 open datasets.
- Model design: a Qwen3-VL backbone is paired with a diffusion/flow-matching action expert and a lightweight world-model expert used only during training.
- System: the robot stack supports teleoperation, deployment, and seamless human takeover via relative action mapping.
- Results: EgoSteer achieved 75% average success across 40 tasks, handled unseen semantic tasks with about 62% success, and learned long-horizon tasks from as few as 120–200 demonstrations.
- Open source: code and weights are already open; data is being opened soon.
More from Embodied
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- Polish developers build iPhone app that detects nearby Meta smart glasses — Low-Honeydew6483 · 2026-09-11
- Ant's Afu health AI hits 150M users, unveils AI+hardware health alliance at Bund Summit — APPSO · 2026-09-11
- Johns Hopkins Launches Full-Stack Hands-on Robot Learning Class with SO-101 Arm Kits — _krishna_murthy · 2026-09-11
- SyncWorld: In-Context Robot World Model Simulates Unseen Views and Embodiments Zero-Shot — ChongZzZhang · 2026-09-11
- A 3D Pose Dataset for Dogs Released — ducha_aiki · 2026-09-11