Action Map Policy turns robot actions into pixel classification, beating Diffusion Policy by 50-70% on real-world tasks
机器之心 · wechat · 2026-09-09
PhD student Haojie Huang (Northeastern University, advised by Robert Platt and Robin Walters) introduces Action Map Policy (AMP), a new action representation for robot manipulation: 3D motion is invertibly converted to 3D keypoint trajectories, projected onto camera planes as 2D pixel trajectories, and prediction becomes pixel-level classification via cross-entropy over keypoint probability maps.
This unifies vision and action in image space (analogous to token space in LLMs), naturally models multimodal action distributions, and outputs the full distribution in a single forward pass. It achieves 1mm positional and 1.3° rotational 3D accuracy at 224×224 resolution, recovering 3D trajectories via multi-view triangulation.
Key results: in a fine-grained laser-pointer test with only 40 demos, AMP hit 100% success while Diffusion Policy showed mode collapse. On three real-world breakfast tasks (coffee, toast, steamed egg), AMP outperformed DiffPo and ACT by 50-70%; inference takes 13.80ms versus 93.53ms for 16-step DDIM DiffPo, suiting real-time closed-loop control.
More from Embodied
- Dev reproduces fly simulation in MuJoCo with visual perception and navigation demo — i_dg23 · 2026-09-09
- PGMT paper: robot tracks flat-ground reference on stairs and 40cm boxes — ChongZzZhang · 2026-09-09
- Humanoid robot learned a real factory task from just 6 demos, ran 4 days at 96% success — lukas_m_ziegler · 2026-09-09
- Maker converts SO-ARM100 robot arm into a biped robot — k7agar · 2026-09-09
- AMD unveils Threadripper Halo workstation for serious local AI — Apprehensive_Bar6609 · 2026-09-09
- Ex-autonomous driving e2e tech lead Zhang Ying launches Physical AI startup, raises hundreds of millions of RMB in a month — 机器之心 · 2026-09-09