RoboColosseum: Hugging Face-backed open benchmark tests robots in 140+ real scenes
DJiafei · x · 2026-11-05
RoboColosseum, an open robot eval initiative backed by Hugging Face, BitRobot, and eaic, deploys 100+ real robots (Unitree G1, LeRobot SO-101, dual-arm YAM, Franka) across 140+ in-the-wild homes, offices, and public spaces, with 10,000 free eval hours.
- Open track: evaluators give fresh on-the-spot instructions; two anonymous models compete head-to-head and the evaluator picks the better run.
- Fine-tune track: every model is fine-tuned on the same 100 demos for a fixed task, then scored over 25 attempts.
- Full leaderboards and all rollout data, failures included, are published on Hugging Face.
The builders argue robotics needs open benchmarks, evals, and leaderboards — a proven way to build better models.
More from Embodied
- A 1X humanoid robot quietly stands in an apartment, teased for upcoming Core Memory feature — kyliebytes · 2026-11-05
- Bambu H3C draws up to 1850W, needs dedicated 20A outlet or it will trip breakers — RexDouglass · 2026-11-05
- Robotics sim frontier names from World Labs, NVIDIA, PI to meet in SF on sim2real gap — jia_seed · 2026-11-05
- Robotics RL still on Isaac Lab PhysX despite mjwarp's 2x+ training speedup — carlosdponx · 2026-11-05
- Widespread broken code and training data in academic robotics projects — carlosdponx · 2026-11-05
- General Intuition: game clips (1.5T action tokens/month) train one model to control 10+ robots — AI Engineer · 2026-11-05