RAX: Robots Learn Fine Manipulation Through Scaled Exploration, Not Demonstrations
Yilun Du's team has released RAX, a robotic system built on a core claim: instead of relying on demonstration data or scaling up model size, robots can learn fine manipulation through scaled online exploration. RAX's key design keeps the underlying model frozen—learning happens in the agent's ever-growing workspace of code, notes, and verified skills: execute a policy, observe the outcome, revise perception and control code, execute again, with each round of exploration building on prior programs, measurements, and findings. Results accumulate as a reusable skill library, where successful decisions are consolidated into persistent skills encoding the full perception-planning-execution pipeline.
Confirmed
- Benchmark results: 78.3% on RoboDojo, 53.4 points above the previous SOTA of 24.9%; 46.5% vs 8.2% on BEHAVIOR-1K long-horizon household tasks, a 38.3-point lead; pouring tasks went from 0 for all prior methods to full marks; over 37.4 points above SOTA across five benchmarks combined
- Cognitive architecture: the agent uses RGB-D and force observations to build task-relevant 3D models, searches for actions via robotics control and kinematics tools, and autonomously develops perception pipelines, manipulation strategies, and execution verification
- Real-robot results: without any teleoperation or action demonstrations, it learned precise peg insertion, air fryer operation, stain wiping, tracking hidden objects, stacking Jenga blocks, and long-horizon whole-body control; inference time dropped from 72 minutes to 18 seconds
- When facing objects like air fryers, the agent autonomously builds their articulated 3D models and plans over them
Why it matters
Robotics has long depended on teleoperated demonstration data and large-model scaling, with high data collection costs and limited generalization. RAX demonstrates an alternative path: freeze the model and simply expand the agent's online exploration budget, achieving large gains on both simulation benchmarks and real robots, with exploration results consolidating into a reusable skill library that compounds capability for future tasks.
2026-10-17 ~ 2026-10-17 · 8 related posts
Primary sources
- RAX scales robot exploration, not demos: zero teleop, +37 points on benchmarks — du_yilun ·
- RAX hits 78.3% on RoboDojo vs prior SOTA 24.9%, scores 100% on pouring where baselines get 0% — du_yilun ·
- RAX learns precision robot skills with zero teleoperation, cuts inference from 72 min to 18 s — du_yilun ·
- [source] RAX scales robot exploration, not demos: zero teleop, +37 points on benchmarks — du_yilun · 2026-10-17
- Frozen model, scaled exploration: RAX beats prior SOTA by 37.4 points — du_yilun · 2026-10-17
- RAX: A Robot Cognitive Architecture That Turns Exploration into Persistent Skills — du_yilun · 2026-10-17
- RAX separates the coding agent workspace from the robot stack for multi-embodiment — du_yilun · 2026-10-17
- One agent, many bodies: how RAX controls humanoids, dexterous hands, and more — du_yilun · 2026-10-17
- [source] RAX hits 78.3% on RoboDojo vs prior SOTA 24.9%, scores 100% on pouring where baselines get 0% — du_yilun · 2026-10-17
- [source] RAX learns precision robot skills with zero teleoperation, cuts inference from 72 min to 18 s — du_yilun · 2026-10-17
1 near-duplicate retellings: du_yilun