RoboColosseum Runs ~20K Real-World Robot Trials Comparing Five Models and Three Agents
micoolcho · x · 2026-11-07
The RoboColosseum project released its first large-scale robot evaluation results, pushing back on whether GPT-6 Astra really outperforms VLAs — arguing a handful of trials in constrained environments can't answer that.
- Scope: 5 robot models + 3 frontier AI agents, 20K trials across multiple robot embodiments and 124-130 diverse real-world sites
- Researcher Yiru Wang says the findings "surprised us," revealing what today's models can do and where they struggle
- The team frames this as just the beginning of what evaluation at this scale can uncover; full results in a thread
More from Embodied
- Figure humanoid robot strolls through company HQ, nearly indistinguishable from a human — Distinct-Question-16 · 2026-11-07
- CoRL robotics conference sells out with hundreds on the waitlist ahead of Austin edition — tomaszbednarz · 2026-11-07
- Open-source AI wearable ring museo opens pre-orders at $144, works with any agent — bihanmahadewa · 2026-11-07
- InstantHMR open-sourced: real-time 3D human mesh recovery in 5.5ms per person on phone NPU — PurzBeats · 2026-11-07
- Replacing hand-tuned RL rewards with a decision model to teach humanoids to walk — tw_killian · 2026-11-07
- Muse starts rolling out to Meta Glasses, targeting hands-free AI agent control — armand_ruiz · 2026-11-07