RoboColosseum launches a real-world robot benchmark: 30+ policies blind-tested on hundreds of robots across 150+ scenes
micoolcho · x · 2026-11-05
RoboColosseum is a real-world benchmark that runs robot policies blind on hundreds of actual robots across 150+ scenes instead of cherry-picked demos, publishing every rollout (failures included) on Hugging Face.
Two evaluation tracks
- Open track: two unnamed models get the same brand-new instruction in the same scene; a human judge picks the winner or calls a tie — LMArena-style, but on real hardware, measuring zero-shot generalization
- Fine-tune track: every model is fine-tuned on one fixed task from the same 100 open-source demos, then gets 25 attempts — an apples-to-apples skill-learning test
Scale: 30+ policies from OpenAI, Anthropic, NVIDIA, Physical Intelligence, Galaxea, Ai2 and more, tested on Unitree G1, Franka, YAM and SO-101 embodiments.
Leaderboard highlights (screenshots): on G1 no model achieves any full success (top LingBot-VLA-V2 and NVIDIA GROOT-N17 at 0% full success, 23% progress); on YAM, Physical Intelligence's PI05 dominates with 94% full success and 97% progress.
The platform offers free evaluation (advertised 10,000 free hours) and releases all data in LeRobot format.
Related event: RoboColosseum Launches Open Real-World Robot Evaluation with 100+ Robots(60 posts)→
More from Embodied
- China's embodied AI market hits ~$136B in 2025, up 20.4% YoY, report says — Icy-Shift-7557 · 2026-11-05
- OpenClaw robotic arm works as release manager, cherry-picking thousands of PRs — vincent_koc · 2026-11-05
- PyRUA-REAL: coding agents give robots recursive self-improvement in the real world — Qiang Wang · 2026-11-05
- Europe's robot-dense industry eyes grid inspection, but regulation stalls deployment — ingliguori · 2026-11-05
- Bambu Lab launches TPU 75A: softest filament yet for 3D-printed wearables — vincent_koc · 2026-11-05
- VAMOS hierarchical VLA lets a robot self-navigate 2.35 km through Zurich — KyleMorgenstein · 2026-11-05