Recursive Harness Self-Improvement
Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, Yujin Tang
cs.LG, cs.AI
2026-07-17
RHI treats a multi-agent harness as a text prompt and refines it by comparing only against its own previous revision; on 30 open-ended ML research tasks a few iterations beat same-family max reasoning effort, and on opus-4.8 cut inference cost 60% versus the built-in workflow.
Today's strongest coding agents depend on more than the base model. They depend on the harness, the orchestration that assigns roles, sets instructions, decides what passes between subagents, and structures the workflow. Harness and model now co-evolve: a stronger harness yields higher-quality execution traces, and those traces can train future models. The catch is that provider-built general-purpose harnesses must serve everyone, so continual updates are slow and expensive. User-built, task-specific harnesses can be optimized per task, but how to optimize them lightly and converge fast is largely unexplored. The paper asks whether a few cheap iterations can tune a user harness to beat simply raising reasoning effort.
RHI makes two core choices. First, it treats the harness as a prompt-level object, not executable code: agent design (roles, instructions) and workflow (contracts between subagents, orchestration hops) become a text specification, and optimization edits that text. Second, it drops population search for trajectory-local comparison: each round compares the harness only with its own previous version.
The loop: at iteration i the agent solves the task with harness H(i) to produce output(i); an evaluator judges output(i) versus output(i-1) as a pairwise preference and stores it in a self-comparison history; an LLM harness optimizer reads that history (without seeing the evaluation criteria) and rewrites H(i) into H(i+1). Because the harness space is discrete text, there are no real gradients, so the accumulated preference history acts as a momentum-semantic signal. Per-iteration cost is Θ(1), one execution plus one pairwise eval, versus Θ(m squared) for population search.
Why prioritize workflow over roles? The hypothesis: a task-relevant contract makes a subagent pass only what downstream needs instead of the full interaction history, which is a task-specific sparsity pattern on inter-agent communication and trims redundant context and KV-cache use.
The benchmark is 30 synthetic open-ended ML research tasks (quantitative finance, robotics, pharmacy, 10 each), each producing a full code repository with standardized deliverables, scored by LLM-as-judge pairwise comparison across six criteria, with two judge configs and three seeds. Base models are Claude Sonnet 4.6, Opus 4.7, and Opus 4.8, all at high reasoning effort, plus RHI.
Three findings. First, a few RHI rounds break the same-family test-time-scaling ceiling: sonnet-4.6-high plus two RHI rounds beats sonnet-4.6-max in 20 of 30 tasks; opus-4.8-high plus two rounds beats xhigh, ultracode, and max. The hardest result: it beats opus-4.8-ultracode, the built-in dynamic multi-agent workflow, showing a task-specific user harness written into the prompt can outperform a provider-built system workflow. Second, the gains are not from longer output: for sonnet-4.6 and opus-4.8, output-token counts stay nearly flat (normalized 1.42 to 1.86) while scores rise. Third, cost drops: cache read/write is the main expense, and RHI trims it by tightening context; opus-4.8-high plus two rounds costs 60% less than ultracode and cuts cache read/write 64%.
For anyone running coding agents daily, this points to a cheaper path than blindly raising reasoning effort: instead of letting the model think longer, reorganize how its components collaborate for the task. The optimization lives in the prompt, touches neither model nor backend, and converges in a few rounds, which fits continual lightweight specialization across many open tasks. It also backs a larger claim, that future progress will come more from harness-plus-model co-evolution than from bigger models alone.
The honest boundary: this optimizes a separate harness per task, and the paper does not test whether that harness transfers. It also cannot replace train-time scaling; sonnet plus RHI still does not catch a plain opus base.
The authors state RHI complements rather than replaces train-time scaling.
Open questions are sizable. All 30 tasks are synthetic ML research coding tasks, a narrow domain, LLM-generated and LLM-judged with no objective ground truth. The headline 60% saving is against ultracode, an expensive baseline; against plain max it is only 23%. Claim 2 (gains are not from longer output) is itself inconclusive on opus-4.7. And the harness optimizer is another large-model call: the Θ(1) accounting covers in-loop cost, but each round still runs a full agent execution plus judging, and the paper gives no concrete cost figure against a real per-task budget.