Berkeley's Do as I Do turns everyday human videos into dexterous robot hand training data
chris_j_paxton · x · 2026-09-25
UC Berkeley researchers (including Pieter Abbeel, Jitendra Malik, and Mahi Shafiullah) present Do as I Do, an algorithm that reconstructs hand-object interactions from monocular RGB human videos and retargets them into real-world executable actions for multi-fingered dexterous robotic hands.
- Works across egocentric and exocentric in-the-wild video sources, producing robot-complete manipulation data
- Outperforms prior state of the art in hand-object interaction estimation and dexterous trajectory extraction on both ground-truth datasets and online video clips
- Demonstrates 10 real-world motions across varied object geometries and grasps, plus a practitioner playbook for collecting human manipulation data
A RoboPapers podcast episode with the authors is dropping soon.
Related event: Berkeley's Do as I Do Trains Dexterous Hands from Human Video(3 posts)→
More from Embodied
- Robot fails at folding a towel, reassuring roboticists frontier models won't eat robotics yet — chris_j_paxton · 2026-11-12
- Nuro podcast: manufacturing, charging, and regulation gate robotaxi mass adoption — R_D · 2026-11-12
- Cybercab rear-ended by human driver, researcher quips: the problem is the humans — chris_j_paxton · 2026-11-12
- HT-Policy Uses Heavy-Tailed Student-t Regression to Close Robot Policy Gap — chris_j_paxton · 2026-11-12
- NVIDIA GTC Berlin Side Meetup Balloons to 2,000 Signups for a 500-Cap Brewery — lukas_m_ziegler · 2026-11-12
- Lamport Robotics' Pincer1 Robotic Arm Priced at $1,199, Payable Only in SOL or USDC — R4nsack · 2026-11-12