Frontier models can't yet label robot training data: best model only 43.5% fully correct
kimberlywtan · x · 2026-10-13
Concept and MeckaAI tested whether frontier models can label egocentric videos for robot learning, having five models caption 200 clips with human grading. The best performer, GPT-6 Astra, got only 43.5% of clips fully correct. The accompanying chart plots accuracy vs cost per hour of video: GPT-6.1 Sol nears 40% at $10/hr, GPT-6 Astra reaches 40%+ at $30-100/hr, while Claude Opus 5.5, Claude Fable 5.1 and Gemini 3.8 Flash sit at roughly 20-30%.
Related event: Frontier Models Struggle to Label Robot Egocentric Videos(2 posts)→
More from Embodied
- NVIDIA takes RTX Spark laptop launch events global, with a Jensen-signed ASUS ProArt P14 up for grabs — NVIDIAAI · 2026-10-13
- Actuator speed, not just AI, is what's holding robots back from snappy actions — ayushthakur0 · 2026-10-13
- Stellantis demos Fiat 500e with Wayve's self-driving AI on Turin streets — alexgkendall · 2026-10-13
- Wayve CEO Stages at Wave to Push Autonomy into Fiat and Maserati — alexgkendall · 2026-10-13
- Japan's Mujin raises $224M Series D as SK Square bets on robot-run factories — lukas_m_ziegler · 2026-10-13
- Boston Dynamics taps Alexa creator Rohit Prasad as CEO to drive commercialization — lukas_m_ziegler · 2026-10-13