RoboWild benchmark: best frontier model scores just 13.3% on real-world robot use across 341 tasks
taoyds · x · 2026-11-21
Two years after OSWorld, the XLang NLP team launches RoboWild, testing whether frontier models can generalize to robot use in the wild:
- Scale: 38 furnished realistic scenes (homes, stores, workshops, labs), 4,480 objects with operable doors, dials and drawers, 341 tasks across 4 robot embodiments
- Results: Claude Fable 5.2 leads at just 13.3% accuracy; GPT-6 Astra is far more token-efficient but plateaus at 11.6%
- Takeaway: unlike the rapid progress on digital computer-use tasks, physical-world robot agents remain far from solved
More from Embodied
- Dev builds Iron Man to Dr. Doom AR lens switch in one day with Claude — Scobleizer · 2026-11-21
- VR-teleop-kit ported to Ulixarm robot arm by community developer — DominiqueCAPaul · 2026-11-21
- Agentic Robotics Night in Zürich: founders panel on building robots that perceive, reason and act — DominiqueCAPaul · 2026-11-21
- Nous hackathon runner-up Housemate turns old phones into 2.6s-response room voice agents — NousResearch · 2026-11-21
- Why one robot's day of runs proves nothing: chained two-robot reliability testing explained — lavanyaai · 2026-11-21
- Wheel-less Denza Leopard 5 robot tugs a Boeing 747 on airport tarmac — TansuYegen · 2026-11-21