HKU's Taku Komura: Building Embodied AI World Models with Consumer Hardware
量子位 · wechat · 2026-07-31
At SIGGRAPH 2026, HKU Professor Taku Komura received the Test-of-Time Award for his 2016 motion synthesis paper. His research vision is clear: enabling machines to learn human movement structures from data to understand the physical world.
From Motion Synthesis to Physical AI
A decade ago, Taku's team pioneered deep learning for 3D character motion synthesis, introducing the concept of "Human Motion Priors." Since then, his research expanded from isolated pose generation to complex interactions between bodies, objects, and environments. His open-source project, AI4Animation, has over 8,000 GitHub stars.
Breaking the Embodied Data Bottleneck
Teleoperation data collection is expensive and limited. Taku’s team believes the breakthrough lies in consumer-grade scenarios. Leveraging their human interaction priors, the team uses just an iPhone to capture and reconstruct daily operations in 3D, reducing hand reconstruction errors by 60% and cutting costs to a fraction of traditional methods.
Multimodal Native World Models
Current embodied data relies heavily on vision and trajectory, but similar motions can have vastly different forces and contacts. The team is building a human-centric, multimodal native world model: using wearable devices to synchronously capture first-person video, eye-tracking, and electromyography (force and contact). This shifts the world model's prediction target from next-frame pixels to actual physical state changes (e.g., force magnitude, contact position), allowing robots to genuinely understand physical consequences.
More from Embodied
- BMW Tests Figure 03 Humanoid Robots in Spartanburg Plant — emmanuelvivier · 2026-07-31
- Samsung Establishes Robotics Division to Accelerate Industrial and Service Bots — emmanuelvivier · 2026-07-31
- CoorDex Enables 49-DOF Humanoids to Walk and Grasp Simultaneously — 量子位 · 2026-07-31
- Physical Intelligence Clarifies Strategy: Solving Physical Intelligence by Any Means Necessary — ZeYanjie · 2026-07-31
- Live Footage: Grinding of a Large Motor Shaft — MatthewChang · 2026-07-31
- M5Stack Workshop: Training Self-Balancing Robots with RL from Sim to Hardware — ShawnHymel · 2026-07-31