Hugging Face and Partners Launch RoboColosseum Open Robot Eval

RoboColosseum is an open robot eval initiative jointly launched by Hugging Face, BitRobot Network, EAIC, and the MAGIC lab at the National University of Singapore, officially going live on November 4. It targets the industry's "trust me bro benchmarks" problem with the motto "benchmarks on real hardware > benchmarks on slides," aiming to become a global proving ground for physical AI where labs can pit their policy models against each other.

Confirmed

• Scale: access to 100+ real robots, including Unitree G1, LeRobot SO-101, YAM, and Franka; the Unitree G1 serves as the main body for the humanoid benchmark category, and Unitree has announced support for the initiative
• Covers 140+ real-world scenarios, including in-the-wild environments like homes and restaurants
• 10,000 hours of free evaluation quota; researchers submit policies and the team runs them on real robots in real environments
• All rollout results, including failures, are made public
• Hugging Face's own benchmark measurements show that generalist robot policies have weak cross-embodiment consistency

Why it matters

• Large-scale real-robot evaluation is costly, and the free 10,000-hour quota significantly lowers the barrier for research teams to validate their work
• The transparency of publishing all failures provides a comparable, reproducible standardized evaluation path for embodied AI policies, addressing industry concerns about the credibility of self-reported benchmarks
• The finding of weak cross-embodiment consistency sounds the alarm on the generalization ability of generalist robot policies

2026-11-04 ~ 2026-11-05 · 21 related posts

• [source] RoboColosseum Benchmark Finds Weak Cross-Embodiment Agreement in Robot Policies — huggingface · 2026-11-04
• RoboColosseum launches open robot eval: 100+ robots, 10,000 free eval hours — DJiafei · 2026-11-04
• RoboColosseum Launches: 100+ Real Robots, 140 Wild Scenes, Open Data — micoolcho · 2026-11-04
• RoboColosseum launches as an ultimate proving-ground benchmark for robots, with free eval credits — DJiafei · 2026-11-04
• [source] RoboColosseum opens 10,000 free real-robot eval hours with public results, failures included — DJiafei · 2026-11-04
• RoboColosseum goes live to pit robot labs' policies against each other — DJiafei · 2026-11-04
• RoboColosseum launches open robot eval with 100+ real robots, 140+ wild scenes, 10k free hours — DJiafei · 2026-11-04
• RoboColosseum launches to take robot evals out of the lab into the real world — DJiafei · 2026-11-04
• RoboColosseum launches real-world Physical AI benchmark: 100+ robots, 20k+ eval runs — DJiafei · 2026-11-05
• Open robot eval initiative RoboColosseum launches; MolmoAct 2 ranks second behind GPT-6 Astra — DJiafei · 2026-11-05

11 near-duplicate retellings: DJiafei · DJiafei · DJiafei · DJiafei · DJiafei · DJiafei · DJiafei · DJiafei · DJiafei · DJiafei · DJiafei