Dex4D Combines Point-Track Policy and Video-Gen Planners for Sim-to-Real Dexterous Hands
chris_j_paxton · x · 2026-09-05
Researchers led by Yuxuan Kuang (CMU) unveiled Dex4D, a sim-to-real framework for generalizable dexterous manipulation.
Key points:
- AP2AP (Anypose-to-Anypose) task-agnostic policy: casts manipulation as transforming an object from any initial 3D pose to any target pose, with no task-specific structure, predefined grasps, or motion primitives; Paired Point Encoding preserves correspondence and permutation invariance between current and target points.
- Trained in simulation on 3,000+ objects, deployed zero-shot to real robots with no real-robot data collection or fine-tuning.
- High-level planner uses video generation plus 4D reconstruction; works across diverse objects, tasks, scenes, camera views, and perturbations — no parallel grippers, no teleop.
Project page, paper, and code (simulation/vision/hardware) are open-sourced.
More from Embodied
- Tuesday Labs' home robot Voxel charms with Pixar-collaborated personality design — cneuralnetwork · 2026-10-20
- Tuesday Labs opens reservations for Voxel Series-1 home robot, first 999 units — cneuralnetwork · 2026-10-20
- Wrist-worn dictaphone is the best ambient compute form factor, says dev — curious_vii · 2026-10-20
- a16z leads Eka Robotics Series A, betting on Vision-Force-Action models for dexterous manipulation — a16z · 2026-10-20
- Autonomous Lamp: a $999 open-source desk robot giving personal AI agents eyes, ears, and a body — dee_hw · 2026-10-20
- Paid $60 for two humanoid robots to clean for an hour: here's how it went — 2C_ornot2C · 2026-10-20