Eight-task robot RSI averages 91.1%; Ethernet insertion stays online

PyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World

Qiang Wang, Jianxin Bi, Shulong Jiang, Ruiyang Si, Wenbo Huang, Kaiqi Chen, Jiaxin Li, Liyun Li, Xuanyi Li, Daquan Zhou

cs.RO

2026-11-05

PyRUA-REAL turns real Franka exploration into frozen programs: 91.1% mean development success on eight tasks, and 84.42% on RoboDojo versus 50.35% for PyRUA-Lean.

What problem this solves

A language model can already turn a manipulation instruction into robot code and patch that code after a failure, without changing its weights. The gap shows up on the next task. Deployment often means calling the same model again to make the decisions.

PyRUA-REAL runs that loop on a dual-arm Franka. Coding agents, agents that write code, read execution feedback, and edit the code, run the experiments themselves. Interaction is kept as memory and reusable skills, then frozen into a program that does not call a language model at runtime. Calibration and task controllers are not provided up front.

Method

A person supplies a goal and a task specification: which objects, poses, and variants the program must cover. The Researcher owns the approach. For each experiment it sets the objective, the stopping rule, and which past experience to consult. An experiment may stop because it is done, because a hypothesis failed, or because several revisions produced no progress. The Engineer writes code, runs the robot, and diagnoses faults inside that round. A bad detector can be compared on its own step, without replaying the whole task. When the round ends, the Researcher decides whether to continue, change approach, or move on.

Experience has three layers: facts about the cell, a record of attempts and failure causes, and code, both full task programs and smaller primitive functions. A skill is kept only after it is retested in the current setup. The frozen program still perceives, controls, and recovers from live observations. It may call local SigLIP and SAM. It does not call a language model.

Results

Across eight real tasks, development success averages 91.1%. That average includes the online-assisted Ethernet runs. It pools attempts made while the code was still changing, so it is not the reliability of one frozen program. Seven tasks run without a language model, with development success from 91.67% to 100%. Ethernet insertion does not. With no online agent the record is still 0 after about 240 hours. With GPT-6-Astra Medium deciding between motion stages, a success appears after about 15 further hours, and the development rate is 58.82%: 0/4 scripted, 10/17 assisted. Time to first full success runs from about 1.5 hours to about 45 hours.

After the first success, the programs keep changing. On five-block stacking, median cycle time falls from 285.3 s to 156.0 s. Visually estimated mean lateral error falls from 2.69 mm to 0.44 mm, and angular error from 1.37 degrees to 0.25 degrees. Tic-tac-toe success goes from 50% to 100%, with regressions in between. Game-plus-reset time falls from 577 s to 535 s, mostly when a chain of raised waypoints becomes a single joint motion. Tower of Hanoi buys reliability first and speed later. Median cycle time goes from 1402 s in the robust stage to 883 s and then 793 s. Full-cycle success goes from 60% to 100%.

On 42 RoboDojo tasks, averaged equally across five capability groups, frozen programs reach 84.42%, against 50.35% for PyRUA-Lean and 22.48% for GPT-6-Astra. The benchmark scores are 88.53, 55.26, and 28.97. The gain is in precision (74.00% vs 38.25%) and long-horizon execution (56.00% vs 43.75%). Generalization is lower, 41.17% vs 43.17%. The evaluations were run separately, and the seeds are not aligned.

ComparisonMetricResult
Extra reviewerMedian gap until the next hardware trial16.9 min (40 gaps) vs 3.4 min (9 gaps)
Revision feedbackHeld-out success after five revisions on four tasksOutcomes 15.3%; state 45.8%; video plus joints 54.2%
Explicit variant rangesFive held-out categoriesPlus 11.4 to 36.6 percentage points
Prior primitive skillsMean on eight new tasks40.0% vs 21.3%
Frozen deploymentTime versus online executionOnline coding about 4.6 to 5.0 times slower; direct action selection about 9.9 to 11.5 times slower

On Hanoi, 45 hours with a reviewer produced no verified disk-on-peg insertion. Direct testing then inserted a disk 2.8 hours after the first grasp. Sending a stalled approach back to the Researcher lines up with progress: in all three physical cases the next trial succeeds. In simulation the same move is uneven, with post-reassessment records from 4/11 to 11/17.

Optimizer time is 25.4 minutes with video and joint trajectories, and 31.8 minutes with outcome reports only. After agents receive pose ranges and keep exploring, stack-bowls rises from 0 to 86.4%, fasten-screws from 9.1% to 81.8%, and swap-blocks falls from 68.2% to 59.1%. Memory goes from 53.8% to 65.2%, and open tasks from 14.4% to 51.0%. A library of earlier primitives helps five of eight new tasks. Classify-by-language and play-stacking-toy each gain 50 percentage points. Three tasks gain nothing.

The timing comparison leaves out development. On stacking, online coding spends 10.6 of 13.7 minutes waiting on the model. Neither online baseline finishes Hanoi within 120 minutes.

Why it matters

Most of the saved time is time not spent waiting on a model. On tic-tac-toe and stacking, online coding is about 4.6 to 5.0 times slower, and direct action selection about 9.9 to 11.5 times slower.

An extra agent that can only reject code makes new physical evidence arrive more slowly. What is worth changing is an approach that local edits are not moving. Ethernet insertion still will not freeze: 0/4 scripted, 10/17 with an online agent. In one successful run, motion takes 281.8 s and the gaps for checks and decisions take 220.8 s, 44% of 502.7 s. Those gaps include human review.

Task by task, the gains are incremental, and they are not the same gain.

Limitations

The abstract's 91.1% is easy to read as a deployment number. It is a development mix.

The comparisons carry a second change along with the one being discussed. Review comes before direct testing, so the later insertion can use the earlier search. Reassessment comes with more trying. Variant specs come with extra exploration. Each feedback condition is a single development session, and numeric state includes ground-truth object poses from the simulator. RoboDojo numbers come from separate evaluations with unaligned seeds. The introduction says 84.82%; the results section and Figure 8 say 84.42%.

The millimeter errors are visual estimates. On stacking, a large reported return error came from the wrong camera view. Correcting the view lowered the number, without a separate check that the motion itself was more accurate. On tic-tac-toe, two touching rings caused a false rejection. After the detector was fixed, the same image passed, and the arm had not moved.

The loop only improves methods the model thinks to try. In simulation, general pickup improved after a person pointed at SigLIP. With no one available, the Hanoi run learned to push hard-to-grasp disks into reach, and that push later showed up in tic-tac-toe. Hints become part of the skill library. The paper says those human contributions should be reported. It does not define when to step in, or how an online decision gets written back into code. After about ten days without an online agent, Ethernet insertion still has no frozen program.

Terms

Source

Related papers

All paper explainers