RoboColosseum benchmarks robot AI on 100+ real robots; none finished a task on Unitree G1
DJiafei · x · 2026-11-05
Hugging Face, BitRobot Network, EAIC and NUS professor DJiafei launched RoboColosseum, an open evaluation platform that fixes robotics' fragmented benchmarking:
- 100+ real robots (Unitree G1, LeRobot SO-101, bimanual YAMs, Frankas) tested across 140+ in-the-wild scenes like homes and restaurants, starting in Singapore.
- 5 robot models including NVIDIA GR00T and Physical Intelligence π0.5, benchmarked against general-purpose models GPT-6, Opus 5.5 and Grok.
- Key finding: even after 100 training demos, not one model finished a single task on the Unitree G1; general-purpose AI models held their own against specialized robot models.
- Academic labs get 10,000 free evaluation hours, and every rollout—failures included—is published on the Hugging Face Hub.
More from Embodied
- Robotics RL still on Isaac Lab PhysX despite mjwarp's 2x+ training speedup — carlosdponx · 2026-11-05
- Widespread broken code and training data in academic robotics projects — carlosdponx · 2026-11-05
- CoRL paper GUILD: actively guiding robot data collection toward useful auxiliary tasks — RobobertoMM · 2026-11-05
- TAMPAgent at CoRL: coding agents + TAMP planning for zero-shot long-horizon mobile manipulation — dhadfieldmenell · 2026-11-05
- Raven Prism launches: a full Linux computer in glasses, controlled by eyes and voice — Scobleizer · 2026-11-05
- Johns Hopkins hosts 'Embodied AI in Healthcare' symposium on December 14 — mdredze · 2026-11-05