RoboColiseum benchmark: Claude Opus 5.5 tops general MLLMs at 76.28 but trails specialized embodied models
机器之心 · wechat · 2026-10-22
Embodied AI eval platform RoboColiseum ran Claude Opus 5.5, GPT-6 Astra, GPT-6.1 Sol, and Gemini 4 Argon through 78 high-fidelity simulation tasks via its Agent2Act framework.
Results
- Opus 5.5 led general MLLMs at 76.28, with Astra at 75.99, Sol 70.91, Argon 66.86 — but specialized embodied policy models Paimon (77.54) and HiDream-O1-Embodied (76.92) still top the overall board.
- General manipulation remains the gap: MLLMs scored 38.79–46.70 vs 63.20–66.13 for policy models.
- Each model has strengths: Opus leads spatial understanding/manipulation, Astra instruction-following and disturbance robustness, Sol is cheapest, Argon has bright spots like package sorting.
Disturbances & ablations
- Image-quality degradation is the shared weakness: Sol's instruction-following dropped from 95.5 to 18 under visual noise; Opus held 76.
- Removing execution feedback, depth input, or history each cut success from 90% to 80% and multiplied planning steps/time.
Real robot & platform
- Astra completed the same pick-and-place on ARX, AgileX, G1, G2; multi-stage cloth folding still fails on hardware.
- Sim-to-real correlation reported at 89.5% within verified scope; eval submission takes 30 minutes via natural-language AgentSkill.
More from Embodied
- MIT CSAIL's X-Hinges 3D-Prints Everyday Objects With Built-In Sensing Across Three Motion Axes — MIT News AI · 2026-10-23
- Formic raises $50M Series B to scale physical AI across 100+ US factories — garrytan · 2026-10-23
- fal starts fine-tuning a physical robot to draw, results next week — OdinLovis · 2026-10-23
- PEARL demo: an LLM agent that checks experiment results, revises plans, and re-tests — KevinKaichuang · 2026-10-23
- CU Boulder's ATLAS Institute Opens Tenure-Track Robotics Professorship — ryosuzk · 2026-10-23
- MiTaS multi-resolution tactile imitation learning hits 80% success vs 35.5% vision-only baseline — Jan_R_Peters · 2026-10-23