RoboColosseum opens failure logs from 100+ real robots to fix manipulation training data
DJiafei · x · 2026-11-06
Andrew O'Shei argues robot evals need open failure logs, highlighting RoboColosseum — an open eval infrastructure project partnered with Hugging Face.
- 100+ physical robots (Unitree G1 humanoids, LeRobot SO-101s, Franka arms) across 140 environments, with 10,000 free eval hours planned
- Every rollout is published on the Hub, including failures
His argument: physical evals usually hit an economic wall, so the field relies on simulation that ignores contact friction or curated lab runs where only successful grasps make it into write-ups. Manipulation foundation models fail in production precisely because they rarely see boundary conditions during training — when an arm slips or drops an object, curated data offers no recovery path. An open registry of real-hardware failure traces provides the negative examples needed for autonomous recovery, which is worth far more than another leaderboard score.
Related event: RoboColosseum Launches Open Real-World Robot Evaluation with 100+ Robots(76 posts)→
More from Embodied
- Dollhouse Robotics founder headed to CoRL, focused on humanoid locomotion — carlosdponx · 2026-11-06
- Atlas unveils humanoid Atlas 1.0, a robot made for everyday life — Scobleizer · 2026-11-06
- iPhone-only SpatialAI 3DGS captures rival a professional capture rig, dev says 'solved' — risingodegua · 2026-11-06
- DIY walkie-talkies and charms for talking to AI agents are popping up on Reddit — CandyMonty · 2026-11-06
- Chat models beat robot-specific policies in blind tests: Opus 5.5 +12.1pp, GPT-6 Astra +11.0pp — DJiafei · 2026-11-06
- Tesla Roadster Rocket up close: 10 SpaceX cold-gas thrusters, invite-only — beffjezos · 2026-11-06