ViDiHand (NeurIPS Spotlight): pretrained video diffusion models excel at 3D hand motion capture
XingangP · x · 2026-10-13
ViDiHand (NeurIPS 2026 Spotlight) from Xingang Pan's team shows pretrained video diffusion models (VDMs) can be repurposed for 3D hand motion capture, echoing work like GenCeption suggesting VDMs are strong visual learners, not just generators.
Key insight: the ability to generate coherent videos means VDMs implicitly learn motion, structure, and correspondences, which transfer to perception.
Highlights:
- Reconstructs accurate, temporally smooth 3D hand motion from monocular egocentric video, robust under severe occlusion and complex hand-object interaction
- No hand detector, no motion infilling, no test-time optimization
- Substantially outperforms prior methods on three benchmarks
The authors argue video generative models hold great potential as foundation models for 3D perception and embodied data collection.
More from Research
- Documents Aren't Knowledge: AI is starting to shape the human judgment that trains it — jteevan · 2026-10-14
- Correction Training lets diffusion models match CFG quality without guidance — kastnerkyle · 2026-10-14
- DeepMind tests Gemini on protein engineering workflows with promising zero-shot results — alexechu_ · 2026-10-14
- Google touts 90 peer-reviewed disease papers in 5 years as it pushes AI into biomedicine — DeryaTR_ · 2026-10-14
- Caltech's Hirosi Ooguri on whether AI can discover physics before we do — TOEwithCurt · 2026-10-14
- World Mech AI argues interpretability is critical to scaling physical intelligence — soniajoseph_ · 2026-10-14