11,398 real-world evals: GPT-6 Astra stands out in 5-VLA policy comparison
DJiafei · x · 2026-11-05
The RoboColosseum team released a 34-page technical report comparing 5 VLA policies and 3 frontier AI agents in the wild: 41 days, 124 diverse sites, 11,398 real-world evaluations.
Key takeaway: OpenAI's GPT-6 Astra is seriously strong, and its demos on X line up with what the team observed at scale. The report contains many more insights.
More from Embodied
- Robot world model demo shows a grasp that never happened in reality — Negative-Whereas3307 · 2026-11-05
- China demos AI blackboard turning sketches into live 3D models, claimed in 60,000 schools — CurieuxExplorer · 2026-11-05
- Gym-Anything for Robotics uses coding agents to mass-generate realistic robot simulations — wellecks · 2026-11-05
- Maker kicks off a second attempt at building a 6-DOF robot arm, armed with lessons learned — _Stocko_ · 2026-11-05
- Toloka tests GPT-6 Astra for robot control: 90% success vs 30% for classical stack — MParakhin · 2026-11-05
- NYU and Meta FAIR introduce ID-WM, a hierarchical world model beating baselines by 38.5% — ylecun · 2026-11-05