RoboColosseum launches: 5,699 pairwise real-world robot evals across 124 sites
DJiafei · x · 2026-11-05
RoboColosseum scales real-world robot policy evaluation beyond the lab: 5,699 pairwise comparisons across 124 sites, hundreds of robots in 140+ scenes, 5 active embodiments. Evaluators write instructions on the spot; the Open track runs blind A/B comparisons of anonymous models, and the Fine-tune track standardizes on 100 demonstrations plus 25 attempts. All demos and runs (failures included) are published on Hugging Face in LeRobot format, and the project offers 10,000 hours of free evaluation.
More from Embodied
- 11,398 real-world evals: GPT-6 Astra stands out in 5-VLA policy comparison — DJiafei · 2026-11-05
- PyTorch Day Japan agenda: backpropagation for real-time robot inference with ExecuTorch — PyTorch · 2026-11-05
- Unitree G1 Named Primary Humanoid Platform for Real-World RoboColosseum Benchmark — DJiafei · 2026-11-05
- Qwen-powered uni-robot-agent hits 31.39 on RoboDojo, beating GPT-6 Astra at 1/19.5 the cost — dotey · 2026-11-05
- UTokyo team 3D-prints lifelike musculoskeletal robot legs via parametric design — repligate · 2026-11-05
- X-Hinges at UIST 2026: 3D-printable robot hinges with embedded sensors grip your finger — repligate · 2026-11-05