Scaling Online Exploration for Robot Agents
Runhan Huang, Xiangcheng Zhang, Haoyu Chen, Yitong Li, Yen-Jen Wang, Bohan Lyu, Haotian Ye, Chuye Hong, Yixuan Wang, Chloe Su, Haoyu Xiong, Yangzhen Wu, Enze Xie, Travis Manderson, Stefano Ermon, Pieter Abbeel, Antonio Torralba, Dawn Song, Weirui Ye, Yilun Du
cs.RO
2026-10-16
A frozen-LLM coding agent explores robots online and compiles successes into standalone programs, beating the strongest baselines by 37.4 points on five benchmarks with zero demos.
Robot learning has been copying the language-model recipe: scale teleoperation data and model capacity to train VLAs and world action models. Every new robot or task still needs its own teleoperation corpus, and the resulting policies stay bounded by their demonstration distributions, so a new layout breaks them. Under LIBERO-Pro's position perturbation, OpenVLA, π0 and X-VLA succeed in at most 2% of episodes on any suite, and π0.5 in at most 38%. The question RAX poses: can a robot learn the way a coding agent learns an unfamiliar codebase, by acting, observing, and fixing, with no human demonstrations? RAX answers yes, and turns the exploration budget into a measurable scaling axis.
A general-purpose coding agent (a frozen LLM; no parameters are trained, no policy is learned) receives a natural-language instruction, a few goal images, and a budget of exploration episodes, then drives the robot directly through its perception and control interfaces by writing code. No action demonstrations, no hand-engineered primitives, no value-function guidance. The loop has three layers:
Two design choices deserve explanation. First, the interface exposes raw RGB-D, camera parameters, and force signals rather than packaged perception, and 3D world modeling emerges from exploration itself: on the air-fryer task the agent fits an articulated drawer model to its own point clouds, and the predicted basket rim lands 6.5 mm from the post-opening measurement; on a 0.1 mm-clearance peg task where wrist-camera depth is unreliable, the agent probes with the arm itself, reading table tilt and board height from drops in vertical force (24.7 mm against a 25 mm CAD spec). Second, a frontend-backend split: the agent acts only through a tool interface (exposed as MCP tools) while low-level control and safety limits stay in the backend, so a new robot joins by implementing one backend plus interface documentation, with no data collection and no policy training. Exploration parallelizes across agents and environments.
Six benchmarks, five embodiments (single-arm, bimanual, dexterous, mobile, humanoid), with exploration and evaluation strictly separated; final programs run alone on unseen scenes. The average margin across the five benchmarks is +37.4.
| Benchmark | RAX | Strongest prior | Margin |
| RoboDojo (7-task avg score) | 78.3% | 24.9% (VPP2) | +53.4 |
| BEHAVIOR-1K (Q-score) | 46.5 | 8.2 (top challenge entry) | +38.3 |
| DexJoCo (32 episodes) | 96.7% | 65.3% (ME-Dex) | +31.4 |
| LIBERO-Pro (perturbation) | 84.4% | 63.2% (ASPIRE) | +21.2 |
| HumanoidArena | 89.8% | 81.9% (WB-WAM) | +7.9 |
The details carry more than the averages. On RoboDojo's open-instruction pouring task the five strongest learned policies all score zero and RAX reaches 100%; on DexJoCo's bimanual Assembly it scores 96.0 against a best published 28.0. On HumanoidArena, RAX uses three teleoperated demonstrations where every baseline uses a hundred; with the demonstrations' action sequences removed it still reaches 81.9%, matching WB-WAM fine-tuned on 100 action demonstrations per task. On the original LIBERO suites it averages 98.5%, on par with fine-tuned models. Performance rises monotonically with budget, from 52.1% at 2 episodes to 80.7% at 32 on HumanoidArena.
On real robots with zero demonstrations, a Franka FR3 handles five tabletop tasks (tracking a cube shuffled under four cups, loading a banana into an air fryer, wiping a stain, 0.1 mm peg insertion, dismantling and rebuilding a Bandu tower), and an RB-Y1 mobile manipulator composes six skills from earlier sessions into a long-horizon tidying task; after one collision with a cabinet, the agent instituted a rule checking the base's swept footprint before every turn. Cost: the first air-fryer success took 72.3 minutes of inference and 262k output tokens; the compiled program runs a trial in 18 seconds and 2k tokens, a 241x speedup. At matched budgets, RGB-D beats RGB-only by 17.3 points on average.
The cost of onboarding a robot changes from "collect a corpus, then train" to "implement one backend interface," a real reduction for the long tail of embodiments such as dexterous hands and humanoids. What exploration produces is readable, editable, composable code, and inference drops from minutes to seconds, a property policy models lack. The perturbation robustness has a clear cause: programs re-perceive and re-localize every episode instead of replaying trajectories, which is why RAX keeps 76.0% on LIBERO-Pro with zero adaptation. This is a second axis next to VLAs: VLAs spend on data, RAX spends on exploration, and it is usable today wherever tasks are automatically checkable and the robot's interfaces are rich.
Exploration is slow and expensive. Between real-robot episodes a human resets the scene, motion is enabled only while an operator has armed the robot, and BEHAVIOR-1K consumed 128 exploration episodes per task plus a separate review model feeding the agent feedback, a cost the paper never itemizes. Budgets run out: make toast random sits at 47.2% after 64 episodes, hang mugs random plateaus at 33-39%, and without action demonstrations the tasks needing sustained locomotion or whole-body contact drop 10.0 to 16.6 points. Read the tables carefully: DexJoCo's 99.5% is a best-session-per-task figure (96.7% at the uniform 32-episode budget), and BEHAVIOR-1K evaluates an agent-in-the-loop program unlike the fully automated ones used elsewhere. The whole system rides on closed-source coding agents, Codex and Claude Code; how much survives a weaker LLM is not reported.