GPT-6-Astra Scores Far Below Published Results When Robot Benchmarks Aren't Tailored
wzenus · x · 2026-11-20
StarVLA Community built starRoboEval, running 14 embodied simulation benchmarks where the agent only sees camera images, robot state and actions. A generic Native GPT-6-Astra Policy scores far below published reference results on 11 of 12 benchmarks — e.g. 34.5% vs 99.9% on LIBERO and 20.0% vs 68.9% on RoboDojo — showing how much published scores depend on benchmark-specific harnesses.
More from Embodied
- Developer claims he 'solved JEPA': model 30x smaller, 100x faster, trains in 30min on a 3080 — MarcJSchmidt · 2026-11-20
- MongoDB founder builds open-source physical AI platform Viam, eyes 1B+ deployments in a decade — gdequeiroz · 2026-11-20
- Waymo robotaxi spotted driving in downtown Toronto at night — GregCook2011 · 2026-11-20
- RealSense Dual RGB + NVIDIA Fast-FoundationStereo live depth demo hits ~8.5 fps — chrismatthieu · 2026-11-20
- Maker builds a voice-command robot servant that figures tasks out itself — Infinite-Job-5616 · 2026-11-20
- Meta's new Muse Charm AI wearable may cost $300 — TechCrunch AI · 2026-11-20