FULL STORY

RoboColosseum Launches With 100 Real Robots for Policy Evaluation

Hugging Face and partners launched RoboColosseum, an open benchmark evaluating robot policies on about 100 real robots. Unitree soon joined, with its G1 humanoid serving as the main platform for the humanoid benchmark category.

2026-11-04 ~ 2026-11-05 · 2 episodes · 53 posts

Episode 1 · RoboColosseum Launches Open Real-World Robot Evaluation with 100+ Robots (2026-11-04, 51 posts)

RoboColosseum is an open robot eval initiative jointly launched by Hugging Face, BitRobot Network, EAIC, and the MAGIC lab at the National University of Singapore, officially going live on November 4. It targets the industry's "trust me bro benchmarks" problem with the motto "benchmarks on real hardware > benchmarks on slides," aiming to become a global proving ground for physical AI where labs can pit their policy models against each other.

Confirmed

  • Scale: access to 100+ real robots, including Unitree G1, LeRobot SO-101, YAM, and Franka; the Unitree G1 serves as the main body for the humanoid benchmark category, and Unitree has announced support for the initiative
  • Covers 140+ real-world scenarios, including in-the-wild environments like homes and restaurants
  • 10,000 hours of free evaluation quota; researchers submit policies and the team runs them on real robots in real environments
  • All rollout results, including failures, are made public
  • Hugging Face's own benchmark measurements show that generalist robot policies have weak cross-embodiment consistency

Why it matters

  • Large-scale real-robot evaluation is costly, and the free 10,000-hour quota significantly lowers the barrier for research teams to validate their work
  • The transparency of publishing all failures provides a comparable, reproducible standardized evaluation path for embodied AI policies, addressing industry concerns about the credibility of self-reported benchmarks
  • The finding of weak cross-embodiment consistency sounds the alarm on the generalization ability of generalist robot policies

31 more related posts →

Episode 2 · Unitree G1 Becomes Primary Platform for RoboColosseum Humanoid Benchmark (2026-11-05, 2 posts)

Unitree's G1 humanoid will serve as the primary platform for RoboColosseum, a large-scale in-the-wild robot benchmark. Playground has integrated it into its Robot Intelligence Index, releasing first real-humanoid evaluation results in which the fine-tuned G1 failed all 625 attempts.