ULTRA trains Unitree G1 humanoids for autonomous whole-body loco-manipulation with one controller
机器之心 · wechat · 2026-09-25
UIUC researchers present ULTRA (Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation), validated on a Unitree G1 humanoid. Accepted at IROS 2026 and shortlisted for the Mobile Manipulation Paper Award.
Key points:
- One controller, multiple instruction granularities: a single set of policy parameters handles both frame-level motion tracking and goal-driven control where only a sparse target (e.g. box destination) is given.
- Physics-based neural motion retargeting: RL in simulation converts human mocap data into executable robot trajectories that respect dynamics and contact constraints; the same retargeting policy generalizes to new motions and object sizes without retraining.
- Two-stage learning: a teacher policy with full state/reference access is distilled into a student using input-availability masks, then RL fine-tuning adds robustness to perturbations and goal changes.
- Egocentric depth perception: object point clouds extracted from depth images replace external mocap; RL fine-tuning substantially improved success rates with point-cloud inputs.
Authors He Xialin and Xu Sirui are UIUC CS PhD students. Paper and project page are public.
More from Embodied
- NSF FRR robotics meeting workshop: four talks on skill learning and physical intelligence — YuXiang_IRVL · 2026-09-26
- Prediction: DeepSeek V4 Flash-scale real-time robot vision models coming within weeks — zhaoran_wang · 2026-09-26
- Déjà View, a NeurIPS Oral: one looped transformer block matches 3D reconstruction models 8-10x its size — ZGojcic · 2026-09-26
- Places Library debuts: 100 high-fidelity real-world 3D environments for embodied AI — yshan2u · 2026-09-26
- microagi x ElevenLabs: Voice is the most human interface for living with robots — animesh_garg · 2026-09-26
- Andy Matuschak builds Printer Friend: voice-to-paper thermal note device — andy_matuschak · 2026-09-26