Unified hand-object model handles 7 tasks, trained on 100K clips mined from unlabeled video

RoozbehMottaghi · x · 2026-11-17

Researchers present a unified hand-object interaction model with two main contributions: (1) a single model supporting 7 tasks — predicting future hand poses, hand articulation and object poses, inferring object motion from hand motion and vice versa, and generating hand/object trajectories conditioned on goals; (2) a scalable pipeline that extracts 3D object poses, hand poses and articulation from unannotated internet videos, yielding a curated dataset of 100K video clips. Paper and code are available.

Original post →

More from Embodied

Embodied channel →