HuGo Uses LLMs and VLMs to Generate Robot Policies — No Demos or Reward Shaping Needed
m_wulfmeier · x · 2026-10-02
Most robot learning effort goes into collecting demonstrations or hand-engineering task-specific rewards. HuGo takes a different route: LLMs and VLMs act as general-purpose tools for policy generation and verification, requiring only language to define new tasks.
Key points
- LLM/VLMs produce high-level commands executed by NVIDIA's SONIC, with low-level RL controllers
- After extensive sim2real and continued real-world learning, the authors report striking data efficiency
- The same approach works out of the box for real-world adaptation
The team, which experienced the pain firsthand working on robot soccer, says the work was led by seoyeonchoi827.
More from Embodied
- Ant Lingbo Embodied AI Challenge crowns TARS team after 48-hour real-robot hackathon — 新智元 · 2026-11-16
- Tesla FSD fleet tops 7 billion city miles, adding ~500M in under four weeks — XFreeze · 2026-11-16
- Ropedia Releases Xperience 2: 1,500+ Scanned Real-World Scenes Paired With Egocentric Demos — liuziwei7 · 2026-11-16
- Hangzhou's Manycore Tech launches Aholo MS07 3D laser scanner with CHCNAV — teortaxesTex · 2026-11-16
- LeRobot 0.7 adds YAM, reBot B601 and Unitree G1 with NVIDIA SONIC support — mishig25 · 2026-11-16
- MathWorks open-sources a GitHub repo of robotics resources for arms, rovers and drones — lukas_m_ziegler · 2026-11-16