Ant Group uses action-free egocentric human videos to boost robot policy learning

antgroup · hf · 2026-11-17

Ant Group presents Ego Guidance, a mixture-of-transformers architecture that jointly predicts future visual states and actions for robot policy learning. A VLM backbone hosts an action expert trained on labeled robot data and a generation expert, while action-free human egocentric videos (cooking, crafts, everyday manipulation) contribute visual priors via cross-attention without requiring action labels, leveraging pretrained vision-language representations to improve policy learning.

Original post →

More from Embodied

Embodied channel →