Berkeley's Do as I Do turns everyday human videos into dexterous robot hand training data
micoolcho · x · 2026-09-24
A UC Berkeley team (Paliwal, Etukuru, Liang, advised by Pieter Abbeel, Jitendra Malik, and Mahi Shafiullah) introduces Do as I Do, an algorithm that converts abundant monocular RGB human videos into robot-complete manipulation data.
Key points:
- Reconstructs hand-object interactions from egocentric and exocentric in-the-wild video sources
- Retargets these estimates into executable action sequences for multi-fingered dexterous robotic hands, bridging the human-to-robot embodiment gap
- Outperforms prior state of the art in hand-object interaction estimation and trajectory extraction, on ground-truth datasets and web-collected clips
- Demonstrated 10 real-world motions across varied object geometries and grasps
Paper and code are open-sourced, and the team proposes an efficacy playbook for practitioners collecting human manipulation data. A full RoboPapers podcast episode with the authors is dropping soon.
Related event: Berkeley's Do as I Do Trains Dexterous Hands from Human Video(3 posts)→
More from Embodied
- Origami Robotics and PalatialSim unveil real2sim2real Dexterity Factory, 100 open datasets next week — ybisk · 2026-11-11
- Trainable JWST model boosts exoplanet sensitivity 2–15x in Nature paper — bravo_abad · 2026-11-11
- Clean point clouds: Tersus GNSS MVP S1 scans turned into high-quality splats in LichtFeld Studio — janusch_patas · 2026-11-11
- Flashing cheap ESP32 IoT devices with ESPHome for a smart-home upgrade — haydendevs · 2026-11-11
- NYU's Steertuning Lets a Robot Learn a Chore From 5 Minutes of Human Demo — CyberRobooo · 2026-11-11
- Boston Dynamics Atlas demoed with Blender 5.3 — TomLikesRobots · 2026-11-11