Dex4D Combines Point-Track Policy and Video-Gen Planners for Sim-to-Real Dexterous Hands
chris_j_paxton · x · 2026-09-05
Researchers led by Yuxuan Kuang (CMU) unveiled Dex4D, a sim-to-real framework for generalizable dexterous manipulation.
Key points:
- AP2AP (Anypose-to-Anypose) task-agnostic policy: casts manipulation as transforming an object from any initial 3D pose to any target pose, with no task-specific structure, predefined grasps, or motion primitives; Paired Point Encoding preserves correspondence and permutation invariance between current and target points.
- Trained in simulation on 3,000+ objects, deployed zero-shot to real robots with no real-robot data collection or fine-tuning.
- High-level planner uses video generation plus 4D reconstruction; works across diverse objects, tasks, scenes, camera views, and perturbations — no parallel grippers, no teleop.
Project page, paper, and code (simulation/vision/hardware) are open-sourced.
More from Embodied
- Festo's chameleon-inspired FlexShapeGripper adapts to any object shape — DoctorJosh · 2026-09-05
- Voice-summoned robot vacuum you can point at spots shows how fast embodied AI is landing — omooretweets · 2026-09-05
- Robot boxing match streams live telemetry via Foxglove; Agent auto-diagnoses the downed bot — jonstephens85 · 2026-09-05
- Cybercab goes live in Austin as Tesla hires AI engineers across the board — elonmusk · 2026-09-05
- Musk shares Cybercab measurement: just ~40 dB at a cruising 40 MPH — elonmusk · 2026-09-05
- Neural Action Codec: compressing robot actions like audio to train VLA policies — YuXiang_IRVL · 2026-09-05