RoboColosseum Benchmark Finds Weak Cross-Embodiment Agreement in Robot Policies
huggingface · hf · 2026-11-04
Researchers from NUS, UW, Stanford, Hugging Face, and BitRobot released RoboColosseum, a benchmark for evaluating general robot policies at scale in-the-wild.
Key findings:
- Weak cross-embodiment agreement: results across different robot bodies are hard to reconcile
- Limited open-world reliability for general policies
- Large zero-shot vs fine-tuned gaps
Paper, code, models and datasets are open-sourced (robocolosseum.ai / GitHub / HF), led by the MAGIC LAB team to make physical-intelligence evaluation reproducible and scalable.
More from Embodied
- Embodied Agent Arena: Closed-Weight VLMs Lead All Five Robot Task Domains, Open-Weight Models Lag — Haojian Huang · 2026-11-04
- PrimeBot's PrimeSim: a generalist world model simulator trained on cross-embodiment demos — Travis Davies · 2026-11-04
- Physical AI Hackathon in SF Nov 7–8 features Dream-1 world action model and robot arms — Scobleizer · 2026-11-04
- Figure robot office demo looks like a sci-fi movie, says CEO Brett Adcock — adcock_brett · 2026-11-04
- 'Beyond Validation Loss': best pretraining checkpoint may hurt downstream RL fine-tuning — SongShuran · 2026-11-04
- Liber-0 hits 36% zero-shot success on 60 robot tasks using 2.2M hours of human data — CyberRobooo · 2026-11-04