UW-led team unveils Grasping the Future, forecasting 3D hand-object motion from internet videos
mangahomanga · x · 2026-11-17
Researchers from the University of Washington, UNC Chapel Hill, and Johns Hopkins present Grasping the Future, a generative model that jointly forecasts 3D hand and object motion from internet videos.
- Given 0.5s of someone handling an object, it predicts the next 4.5s: wrist paths, finger bends, and how the object moves and rotates
- One model unifies 7 tasks: future forecasting, hands-from-object (and vice versa), goal-conditioned generation, and in-filling missing motion
- Trained on 62,562 3D hand-object trajectories rebuilt from Action100M videos (only 12% passed quality checks), plus 8 motion-capture datasets
- The generative model samples multiple plausible futures from the same past
- Forecasts re-targeted to simulated robot hands via SPiDER execute with 73.8% success vs. 14.6% for kinematic re-targeting
Paper, code, and models are released; dataset coming soon.
More from Embodied
- NVIDIA's LidarVerse simulates any LiDAR sensor in real time with one world model — NVIDIAAI · 2026-11-17
- Jetson GPIOs now drive $2 L298N motor controllers directly, no Arduino needed — chrismatthieu · 2026-11-17
- New Unitree Robotics datasets land on Hugging Face — mishig25 · 2026-11-17
- Enactra debuts embodied-AI simulation platform where every frame ships annotated — Lianhuiq · 2026-11-17
- LG targets Q1 2027 to unveil bipedal humanoid on NVIDIA reference design — xmercury_one · 2026-11-17
- Robotics researcher: edge cases are literally the entire problem for robot deployment — chris_j_paxton · 2026-11-17