Robots Learn Skills by Imagining Them First, Achieving Over 80% Success Rate
imjustnewatai · x · 2026-08-06
A new paper shows that humanoid robots can learn skills by imagining the task process first.
- How it works: Given a single image of a scene and a task description, the system generates a video of a human completing the task. It then extracts hands, objects, and depth to convert into whole-body robot motion tested in reality.
- Self-correction: When an attempt fails, a vision-language model (VLM) diagnoses the calibration error and edits the plan.
- Results: Across four real tasks (handing over an object, opening a box, pouring water, opening a drawer), it averaged an 81.3% success rate.
The setup requires no task-specific teleoperation or recorded human demonstrations, performing almost as well (81.3%) as initializing from a real one-shot video (80.0%).
Related event: RoboReact: Humanoid Robots Learn Skills via "Imagination"(2 posts)→
More from Embodied
- Poki Robotics Raises $100M+ in 4 Months to Build Brains for Home Robots — 创业邦 · 2026-08-06
- Embodied AI Shows Dexterous Manipulation; Skild AI Rumored to Have Insane Talent Retention — E0M · 2026-08-06
- Resurrecting a 2015 AI Assistant with ESP32 and a Rotary Display — viksit · 2026-08-06
- Karpathy's AutoResearch Aims to Break Boundaries in Physical AI — teortaxesTex · 2026-08-06
- EgoHumanoid: Training Humanoid Robots via Egocentric Human Videos — micoolcho · 2026-08-06
- UMD Introduces HumanEgo: Training Robots via Human Demonstration Videos — furongh · 2026-08-06