GPT-6-Astra Scores Far Below Published Results When Robot Benchmarks Aren't Tailored

wzenus · x · 2026-11-20

StarVLA Community built starRoboEval, running 14 embodied simulation benchmarks where the agent only sees camera images, robot state and actions. A generic Native GPT-6-Astra Policy scores far below published reference results on 11 of 12 benchmarks — e.g. 34.5% vs 99.9% on LIBERO and 20.0% vs 68.9% on RoboDojo — showing how much published scores depend on benchmark-specific harnesses.

Original post →

More from Embodied

Embodied channel →