Flex-RSI: Benchmarking Recursive Self-Improvement of Frontier Agents
Jingdong Zhang, Jionghao Wang, Zhengming Yu, Yancheng Cai, Wei Chow, Chris Choy, Zhiyang Dou, Hang Hua, Jiachen Li, Zhixuan Liu, Chen Wang, Songlin Wei, Siyuan Yang, Yu Yuan, Xiaohang Zhan, Yucheng Zhang, Shuhong Zheng, Xin Li, Lingjie Liu, Wojciech Matusik, Zhengzhong Tu, Wenping Wang
cs.AI, cs.CV, cs.RO
2026-11-06
Flex-RSI has agents improve both their solution and their own harness on 24 vision and robotics tasks under a fixed budget; Claude Opus 5.5 leads 18 models at 91.6, best open-weight 49.1.
An agent's performance is never just about the model. The harness around it, the prompts, tools, memory, and control loop, decides what the model sees and what it can do, yet most benchmarks nail that harness shut: MLE-bench and RE-Bench score the final artifact, and much of the recursive self-improvement (RSI) literature targets large-scale model training. The question anyone deploying an agent runs into goes unanswered: given a new task and a fixed budget, can it use task feedback to improve how it works rather than only its answer? Prior RSI work also rarely compares procedures from identical starting points at a matched budget, so claims lack controls. Flex-RSI moves the question into vision and robotics, led by Texas A&M with 13 institutions involved, where every attempt burns real compute and the goal is a better solution within hours to days.
The design separates improving the solution from improving the way the agent works:
24 tasks in six areas: seven Action tasks (RoboDojo, RoboCasa, two LIBERO variants, and more), seven Reasoning tasks (ARC-AGI 1/2/3, physical quantity estimation from video), four Creation, three Reconstruction, two Perception, and one Infrastructure task (accelerating a VAE decoder). 18 models entered, 11 closed-source and 7 open-weight.
The leaderboard scores each model by the share of other models it beats per task:
| Model | Score | Note |
| Claude Opus 5.5 | 91.6 | ran all 20 tasks, best test score on 11 |
| Gemini 4 Argon | 89.0 | ran only 9 tasks |
| Qwen3.8-Flash-Next | 49.1 | best open-weight, fifth overall |
Self-improvement raised the validation score in 150 of 177 runs (85%), median gain 10.4 points. Closed-source models improved in 94% of their runs (median +14.9) against 76% (+8.5) for open-weight. On 3D Tracking the median validation score climbed from 19.4 to 62.2, on Scene from Photo from 7.7 to 38.0. Gains arrive early: the median run banks 90% of its gain by round 3 in Perception and Reasoning, while Action and Reconstruction need rounds 6 and 8.
Four findings:
On cost, a median run burns 49.7M tokens and the rank correlation between relative token use and score is −0.05, so spending more buys no rank. Opus runs at 0.35× the median token use and Argon at 1.66× to land close to it; all six open-weight models on the cost chart score below 50, five of them using at least the median.
For practitioners the guidance is concrete. If your task is one where the agent writes its own program, policy, or pipeline, self-improvement under a budget pays off. Make the feedback a continuous score rather than pass/fail; open-weight models are especially sensitive to that. The paired runs give same-start, same-budget causal evidence that harness editing helps, on a small sample of seven pairs. The negative result is just as useful: a stronger model's harness does not transplant. Weaker models starting from a stronger model's final harness improved on only two of four tasks and never approached the stronger model's score. Harness improvement is coupled to model capability; you cannot borrow it.
The authors draw the boundary themselves: all measured improvement sits at the harness level, weights stay frozen, and most tasks train nothing, so Flex-RSI does not test a model modifying its own parameters; weight-level self-improvement is future work. Reading beyond that: each model runs once per task, so there is no multi-seed variance and hairline gaps (0.1 on Video Quality) carry little meaning; coverage is sparse, with Argon on 9 of 20 tasks and the overall score an average over uneven task subsets; the paired and transfer experiments cover seven pairs and four tasks; and ARC-AGI-3 and Scene from Video barely improve (median 3.8 to 4.1), so the loop can fail outright under the strictest checks without a full account of why.