10 Frontier Models Tried to Drive a Surgical Robot; All 213 Success Claims Were Wrong
Exp_Mark · x · 2026-11-11
micro1ai released Dual-Arm Sim, a new robotics benchmark testing how well frontier models control dual-arm embodiments in a simulated operating room, across 800 trials with 10 models.
- Only two models ever completed the task: Gemini 4 Argon 9 times and GPT-6 Astra twice.
- Models claimed success 213 times, and every claim was wrong; in 94 cases the robot never touched the object.
- Scored step-by-step against simulator ground truth on 80 simulated surgical tasks, Claude Fable 5.2 led with a 59.8% progress score versus 49.5% for the next best.
The takeaway: models' self-reported success is wildly unreliable, and there is a large gap between embodied control ability and self-evaluation.
More from Embodied
- Humanoid Factory Deploy Stalls: Teleop Intervention Every 6 Minutes After 15 Weeks — ZeeshanZiaML · 2026-11-11
- CoRL 2026's wildest moments: 'Why do you care about delay?' and 'haptic doesn't matter' — breadli428 · 2026-11-11
- JKU Linz's new Institute for Machine Intelligence hires 2 PhD students in robot learning, deadline in a week — HildeKuehne · 2026-11-11
- Darpinian: Tesla should wait for the AI5 chip before scaling Cybercab production — Darpinian · 2026-11-11
- Maestro paper: cheap Gemini 2.5 beats π₀.5 baselines at zero-shot robot control — dineshjayaraman · 2026-11-11
- OpenDDE: open-source all-atom drug discovery engine hits 51% on hard antibody-antigen benchmark — teortaxesTex · 2026-11-11