XDOF Benchmark: Top LLM Agents Score Just 9-19% on Physical Tasks
mihdalal · x · 2026-10-22
A benchmark table for XDOF shows four LLM agents — Astra, Fable, Sol, and Opus — performing poorly across real-world physical tasks, with overall success rates of just 17%, 9%, 19%, and 12% respectively.
- Tasks include retrieving out-of-reach plush toys, standing up fallen bottles, picking up bread with tongs, sweeping trash, tying knots, and stacking blocks
- The easiest task, retrieving a distant plush, tops out at 86% (Fable); most tasks sit below 20% success
- Several tasks — organizing tableware, serving cereal, tying a knot, unfolding a t-shirt — scored 0% across all four agents
- Setup: up to 30 minutes per agent, 25-30 scored runs per task
The poster notes Argon is missing from the table. The results underline how far current LLM agents remain from real physical manipulation competence.
More from Embodied
- GPT-6 Astra leads Gemini 3.8 Flash 1,349 to 972 on physical agent Elo index — qinzytech · 2026-10-22
- Musk: Tesla needs 10,000-20,000 Optimus bots doing self-play to bootstrap training data — XFreeze · 2026-10-22
- Riding the first production Mercedes shipped with NVIDIA's full self-driving stack — HarperSCarroll · 2026-10-22
- Musk: Austin Cybercab fleet grew 10x in a month as autonomous miles scale exponentially — XFreeze · 2026-10-22
- CMU paper defines solver-sufficient symbols to cut hybrid robot planning cost — Yiyuan Pan · 2026-10-22
- Samsung Next, NVIDIA and CoreWeave to host robotics founder night at CoRL 2026 in Austin — wandb · 2026-10-22