Singapore's RoboColosseum runs 20,000 real-robot trials to benchmark physical AI models
DJiafei · x · 2026-11-06
RoboColosseum, built with Hugging Face, the BitRobot Foundation, the Singapore Embodied AI Centre and NUS's MAGIC Lab, is shared infrastructure for evaluating physical AI on real robots. It has already run 20,000+ physical trials across 6,800 tasks in 150 real-world scenes, using 100+ robots on four platforms from arms to humanoids. Early findings: which model wins depends on the robot — on a Unitree humanoid, no model completed any of five tasks. The initiative aims to make robotics testing repeatable and transparent, shifting from simulation to real-world benchmarks where someone else picks the task.
Related event: RoboColosseum Launches Open Robot Evaluation on 100+ Real Robots(73 posts)→
More from Embodied
- Chat models beat robot-specific policies in blind tests: Opus 5.5 +12.1pp, GPT-6 Astra +11.0pp — DJiafei · 2026-11-06
- RoboColosseum opens failure logs from 100+ real robots to fix manipulation training data — DJiafei · 2026-11-06
- RoboColosseum launches: 100+ real robots, 140+ scenes, 20k evals, all rollouts open — DJiafei · 2026-11-06
- Tesla Roadster Rocket up close: 10 SpaceX cold-gas thrusters, invite-only — beffjezos · 2026-11-06
- PHOS: A Raspberry Pi Robot That Gives Claude Desktop Sensors and Eyes via MCP — jfrankie · 2026-11-06
- Tesla Robotaxi crosses 4.5M autonomous miles with zero injuries and no safety monitor — XFreeze · 2026-11-06