ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
Siwei Wu, Jincheng Ren, Yizhi Li, Haau-Sing Li, Chengran Yang, Yuxuan Zhang, Weicheng Gu, Jian Yang, Riza Batista-Navarro, Chuanyi Zhang, Xianglong Liu, Ming Zhou, Bryan Dai, Chenghua Lin
cs.CL
2026-09-14
Five independently evolved harness modules, trained on 240 held-out tasks, lift TerminalBench Acc from 47.57 to 52.43; joint or whole-harness evolution falls below baseline.
CLI agents are increasingly limited by the harness, not the weights: the loop, the tools, observation handling, context, and the stop condition. A recent wave of Recursive Self-Improvement (RSI) lets an agent edit those execution mechanisms from its own traces. Most of that work evolves the harness on the evaluation benchmark or a subset of it, so a score bump can be benchmark memorization. A single trajectory also mixes systematic harness bugs with task-specific solutions. Rewriting the whole harness at once then hides which component deserved the edit.
ModularRSI, from IQuest Research, the University of Manchester, Beihang, and collaborators, keeps evolution data fully disjoint from downstream benches, contrasts successful and failed traces of the same task, splits the editable harness into five modules, evolves each one in isolation, then merges.
The starting point is Terminus-2 inside Harbor. Sandbox setup, parallelism, and LLM I/O stay frozen. Only behavioral machinery is editable, in five modules: Agent Loop (reason-act-observe and recovery), Tool Use (select, call, validate), Observation Management (filter and compress environment feedback), Context Management (retain, compress, retrieve across steps), and Task Completion Detection (decide whether to stop).
Each evolution instance is rolled out K times and bucketed as all-success, all-fail, or mixed. Mixed tasks pair a success with a failure of the same instance and look for function-level differences. All-fail tasks first search a trajectory memory for an older success; if none exists, they get a one-sided diagnosis. All-success tasks are mined only for wasted steps. Diagnoses in a batch become JSON and are ranked by how many distinct tasks point at the same function. After an edit, three gates run: static checks (AST, imports, protocol), a diff review by the Code-Modify Agent for task-specific hacks, and a live run on two random tasks from the current batch. After all five modules finish, one integration epoch removes duplication and clarifies responsibilities. Similar functions are merged, and at runtime only task-relevant functions are switched on.
Evolution tasks are built from GitHub, Hugging Face, Kaggle, and Linux kernel docs, using only high-level category labels from the TerminalBench and SWE-Bench families. Benchmark instances themselves are never used. Filters cover environment completeness, non-triviality, and whether the evaluator tests functional correctness; a reference solution.sh must pass and a no-op must not. After human review and semantic-similarity screening against the benches, 2,000 tasks remain. The main run samples 120 per side (240 total) for 3 epochs. The workhorse models are DeepSeek-V4-Flash-Preview and DeepSeek-V4-Flash-0731.
Evaluation is on TerminalBench 2.0 (89 tasks) and SWE-Bench Verified (500), with the harness frozen. On DeepSeek-V4-Flash-Preview:
| Evolution set | TerminalBench Acc | SWE Acc |
| No evolution | 47.57 | 73.40 |
| TB-related (in / out of domain) | 52.43 | 75.80 |
| SWE-related (out / in domain) | 49.40 | 76.45 |
In-domain TerminalBench Pass³ (all three independent rollouts must succeed) rises from 30.34 to 35.96. The same TB-evolved, frozen harness still helps other models: GLM-5.2 accuracy 59.55 to 61.80, MiniMax-2.5 41.57 to 44.94, DeepSeek-V4-Flash 47.57 to 52.43.
Modularity is doing real work. On TerminalBench, non-modular evolution falls to 46.44 and joint all-module evolution to 44.19, both below the 47.57 baseline; independent evolution then merge reaches 52.43. Every single module is positive: Agent Loop has the best accuracy (50.56), Observation Management cuts mean steps from 34.70 to 22.50. Against Meta-Harness and AHE on the same 120 tasks, the same Terminus-2 start, and web search disabled (controls run 16 epochs to match total rounds; backbone is DeepSeek-V4-Flash-0731), the baseline is 61.79, the two controls move about one point, and ModularRSI reaches 67.42. A medium-centered difficulty mix scores 76.45 on SWE-Bench; a hard-plus-easy mix scores 74.25.
The paper gives harness RSI a cleaner evaluation contract: evolution data and the leaderboard must not overlap, or a gain is uninterpretable. For anyone writing a coding-agent scaffold, the operational lesson is specific. Do not rewrite the whole loop in one pass. Split by function, localize with same-task success/fail pairs, gate for overfitting and executability, then merge. Observation compression buys steps. The loop buys success rate. Those are different currencies.
This is still incremental engineering. Weights never move; the execution shell does. Transfer across models suggests the edits look more like an interaction protocol than a model's verbal tic. The lift is a few points, not a generation jump.
The authors did not run a dedicated ablation of contrastive trajectory analysis, and the main evolution used only 240 of the 2,000 curated tasks.
Baselines differ sharply across backbone versions (47.57 on Preview, 61.79 on 0731), so those tables cannot be read side by side. The execution gate samples two tasks per edit, and the overfitting review is the same Code-Modify Agent reading its own diff; both gates are soft. Non-modular and joint evolution falling below baseline shows that unconstrained edits hurt, and that part of the modular gain is simply editing less, more locally. Category labels for the evolution set still come from the TerminalBench / SWE-Bench families, so instance overlap is blocked, domain prior is not. Code and data are public; wall-clock time and token cost of evolution are not reported.