Robotics LLMs tested on 16 tasks: GPT-6 Astra hits 17.2% success vs Fable's 9.1%
pabbeel · x · 2026-10-16
XDOF put LLM-driven robots Astra (GPT-6) and Fable 5.1 to the test across 16 tasks covering 13 capabilities.
Results (task-balanced, 95% bootstrap intervals):
- Astra: 17.2% success (79/459 scored runs)
- Fable: 9.1% (43/472 scored runs)
Both did surprisingly well on reasoning and planning but struggled with dexterity; Astra performed better overall.
More from Embodied
- Volvo and NVIDIA expand partnership to bring safe automated driving to next-gen cars — nvidia · 2026-10-16
- Tevel's robotic system harvests fruit around the clock using computer vision — LexiLove · 2026-10-16
- SHOES: 10 minutes of real-robot RL lets quadrupeds walk in shoes, CoRL 2026 paper — kscottz · 2026-10-16
- Tesla Cybercab robotaxis logged zero crashes in first two weeks of public rides, NHTSA data shows — elonmusk · 2026-10-16
- WEDGE paper generates training environments conditioned on navigation agent failures — _akhaliq · 2026-10-16
- Viral meme nails AI's irony: it can write symphonies but can't do the dishes — briannekimmel · 2026-10-16