TCP4D: unified TCP tracking-and-generation model for robot manipulation, adapts with 10 demos
Haosong Peng · hf · 2026-11-18
TCP4D uses an embodiment-agnostic tool-center-point (TCP) action space and a shared 4D scene representation to both track observed TCP motion and generate future TCP trajectories. From monocular video and initial 2D TCP queries, the tracker recovers metric SE(3) poses and gripper states; the generator samples the same 4D features with robot observations and language instructions to predict future TCP states.
- A cross-embodiment augmentation pipeline retargets existing RGB videos into demonstrations for new embodiments without recorded source-robot states
- Tracking: 7.60mm / 16.07mm absolute trajectory error on DROID / RoboLab; adapts to a new embodiment with only 10 demonstrations
- Generation: 69.19% mean success on RoboTwin, beating established policy models on two real-world tasks
- Joint training on source + augmented demos boosts target-robot success by 41.5% over augmented-only training
More from Embodied
- RoboDojo sim "not even wrong": blog exposes physics flaws in popular robot benchmark — ZeYanjie · 2026-11-18
- NVIDIA-backed Agile ONE SN humanoid robot for researchers ships early 2027 — xmercury_one · 2026-11-18
- Peking University Open-Sources PanDWM, a 360° 4D Driving World Model Beating DiST-4D — PekingUniversity · 2026-11-18
- Humanoids serve ice cream and spar in Shenzhen while APEC leaders visit — TansuYegen · 2026-11-18
- Otto by Rockwell explains using Gazebo digital twins for AMR deployments — ___Mufasaa · 2026-11-18
- Tokkey debuts thumb-sized AI hardware to run local models on your Mac — TejasKumar_ · 2026-11-18