RoboQuest benchmark: best AI agent solves only 23.2% of embodied exploration tasks
omarsar0 · x · 2026-11-23
NTU researchers released RoboQuest, a benchmark for goal-directed embodied exploration where agents must search, inspect, and interactively test their environment before committing to a task — 10 mobile manipulation tasks spanning three forms of uncertainty.
- Five frontier multimodal agents were evaluated through a common visuomotor interface; the best succeeded in only 23.2% of episodes, and a π0.5 policy fine-tuned on released full-episode demonstrations almost never succeeded.
- Key contrast: the same models completed the underlying motor skills 72–81% of the time when hidden information was supplied — the bottleneck is evidence gathering and decision-making, not execution.
- Failure analysis: agents stop exploring too early, commit prematurely, and rarely prevent or repair disturbances caused by their own exploration; trial-and-error learning remains hard.
- Takeaway: reliable physical agents need better evidence tracking, planning, and recovery. Paper and demonstrations are open-sourced.
More from Embodied
- Codex Can't Route PCB Traces: Devs Still Lack a Workable AI Tool for Hardware's Last Mile — goldvenetianmask · 2026-11-23
- AI Agent Designs Complete RISC-V CPU Core From Scratch in 12 Hours — ChuckDBrooks · 2026-11-23
- FrankenRobotics: an AI-agent-run robotics lab that batch-designs 256 robots per GPU run — doodlestein · 2026-11-22
- Persona AI's welding humanoid trains with real welders in Korea, commercial site by year-end — CyberRobooo · 2026-11-22
- Beyond motion tracking: why robots still need closed-loop S0 controllers — animesh_garg · 2026-11-22
- One chef, multiple auto stir-fry machines: automated wok kitchen goes viral — churchkey · 2026-11-22