DeepMind robotics evals: Astra overtakes Gemini by ~8%, still 10x error gap to internal models
m_wulfmeier · x · 2026-09-10
A DeepMind robotics engineer updated the team's robot deployment benchmarks with fresh numbers on frontier models:
- Astra has finally overtaken Gemini, jumping nearly 8% since the Sol checkpoint — though its rival is only the Gemini flash model, priced at $10/50 per million tokens vs Gemini's $0.75/3.75.
- Gemini has now surpassed the dedicated Gemini Robotics ER 2 model.
- A large gap remains to the team's internal models, which show roughly 10x lower error rates.
- Credit for keeping the perception evals running goes to Michał Grzejdziak-Zdziarski and team.
More from Embodied
- ROS names Alejandro Hernández Cordero new project lead, eyes 2027 release — ___Mufasaa · 2026-10-27
- ROS Meetup coming to Google's NYC office Oct 29 with speakers from Tangent, Cohesive Robotics, Intrinsic — kscottz · 2026-10-27
- Tesla officially expands robotaxi service area across much of Houston, Texas — Polymarket · 2026-10-27
- Walden Robotics teaches robots to call for help, blending remote assist with autonomy — adnothing · 2026-10-27
- Folding-screen hinges: dozens of ultra-precise parts, one grain of sand can kill the panel — BenBajarin · 2026-10-27
- No π0 weights changed: Motoniq layer lifts real-robot precision fitting from 0/10 to 9/10 — hbouammar · 2026-10-27