HumanoidToolBench benchmarks humanoid tool use from selection to mobile execution on real Unitree G1
snupilab · hf · 2026-10-15
Seoul National University, UMass Amherst and Google Research introduce HumanoidToolBench, the first benchmark jointly evaluating tool selection and mobile execution on humanoids.
- 18 tasks across 3 scenarios, 3 execution levels and 2 tool-set modes, plus ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1.
- Evaluating 7 policies in simulation and 3 on the real robot reveals substantial gaps between selecting a suitable tool and completing the task; GR00T N1.7 probes show reduced selection accuracy on unseen tools and continued execution under unrelated instructions.
- Code and data are available.
More from Embodied
- Gracia's new world model targets Real2Sim2Real and Physical AI — jonstephens85 · 2026-10-15
- Niantic Spatial built VPS on Pokémon Go imagery; a dev already prototyped it solo — jonstephens85 · 2026-10-15
- RoboParty launches RP1, a fully open-source full-stack humanoid robotics platform — GuanyaShi · 2026-10-15
- NYU's Temporal Flow Policy adds physical time to robot memory, beating RNN and state-space memory designs — newyorkuniversity · 2026-10-15
- G³VLA injects calibrated multi-camera geometry into VLA models to improve manipulation — Yue Peng · 2026-10-15
- Survey: Towards Agentic Robots organizes the field into 17 categories across four domains — Chonghan Qin · 2026-10-15