Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training
Kangning Zhang, Wenxiang Jiao, Bowen Fang, Shuai Shao, Jiachen Zhu, Rong Shan, Shijian Wang, Hao Wang, Yuan Lu, Weiwen Liu, Weinan Zhang
cs.AI
2026-11-03
Only all-fail tasks get a sibling-harness rescue. GRPO on those rollouts trains one policy; back on the base harness, 4B beats 9B H0-rescue GRPO on AppWorld, SWE-Gym, and DBBench.
An agent's score depends on the policy and on the harness around it. The harness is the execution layer: prompts, tool interfaces, context and memory, and the rules that turn decisions into actions. Variants grown from one base, H0, share the execution foundation, environment interfaces, and verifier. They differ only in how they support the policy. The base in this work is DeepSeek Harness. The variants are sibling harnesses. The question is whether support from those variants is still there after the policy returns to H0.
On Qwen3.5-4B and AppWorld, evolution from a shared base shows three regularities. Locality: an edit often repairs one slice of failures and regresses tasks the base already solved. Complementarity: siblings recover different errors. An oracle union, one execution under each variant, covers more tasks than any single variant and more than H0 pass@K at the same rollout count. Nonstationarity: as the policy trains, a harness that beat H0 can fall behind it, and rankings flip across checkpoints.
A universal harness is a poor place to put those local repairs. The execution requirements can conflict, and edits that fit in one design still interfere. Alternating harness and policy updates is expensive for the same reason. Old repairs stop helping the new policy, so variants have to be regenerated and revalidated, and gains of the coupled pair need not survive a return to H0. Harness-S uses siblings to rescue base-harness failures, then trains the resulting behavior into one policy.
The sibling pool is built offline and then frozen. Directional Harness Evolution starts at H0. Each generation runs the current parents on fresh training tasks, groups root causes that show up across several failures, and edits only that cluster. One child stops guessing field names and reads the schema first. Another stops after the first page and keeps paging to the end. At most B children become the next parents. Selection maximizes joint validation coverage and counts each solved validation task once, so the parents are the ones that plug each other's holes. Every executable, protocol-compliant child enters the Sibling Harness Pool, whether or not it becomes a parent.
Complementary Harness Rescue Training calls the pool only for tasks H0 cannot solve. The current policy first draws K rollouts under H0. The method figure and the case table use K = 8. If any rollout succeeds, that group is kept. If every rollout fails, the router picks k sibling harnesses. The same policy rolls out once under each, from the task's initial state. That rescue group replaces the all-fail group, and GRPO updates the shared policy. GRPO turns within-group reward differences into advantages. An all-fail group has nothing to compare. A rescue group that mixes successes and failures does. The update touches policy tokens only, in the harness context that produced each trajectory.
The Adaptive Harness Router does not try candidates on the task before choosing. A Harness Behavior Profile states when a sibling helps or hurts the current policy relative to H0. An LLM summarizes paired validation trajectories into conditions and behaviors. Several LLMs score candidates independently from the task, the profiles, validation success, and outcomes on similar tasks. Higher ranks get higher weight, unlisted harnesses get zero, and the top k are selected. Because utility drifts, policy-aware profile re-anchoring freezes the current policy at checkpoints, reruns validation under each sibling and under H0 with matched tasks and budgets, and rewrites the profiles. Test outcomes stay out.
At inference, Rank-1 reuses the same scores. Before any rollout it picks one harness from the full pool, including H0. The executor rollout count matches always using H0.
The policies are Qwen3.5-4B and Qwen3.5-9B. The benchmarks are AppWorld (application APIs), SWE-Gym (software engineering), and DBBench (databases). Pass@3 is the fraction of tasks solved at least once in three rollouts. Avg@3 is mean success across those three. Main numbers average three independent runs, three executor rollouts per task, on eight H800 GPUs. Controls are plain GRPO, GRPO that resamples H0 after an all-fail group (H0 rescue), GRPO mixed with Claude Code or a fixed Top-4 of siblings, frozen-policy Meta-Harness, frozen Harness-S (HE only), and the joint methods WHALE and HarnessForge.
Back on H0, the 4B Harness-S policy scores:
| Benchmark | Harness-S | GRPO | H0 rescue |
| AppWorld Pass@3 / Avg@3 | 67.56 / 56.87 | 60.31 / 48.98 | 61.83 / 51.91 |
| SWE-Gym Pass@3 / Avg@3 | 38.75 / 28.75 | 24.38 / 22.50 | 25.63 / 23.33 |
| DBBench Pass@3 / Avg@3 | 73.33 / 71.00 | 66.67 / 64.00 | 67.33 / 64.67 |
Harness-S reports the standard deviation across the three runs. AppWorld Pass@3 for 4B on H0 is 67.56±1.87.
The 9B policy on H0 also beats same-size GRPO. Pass@3 / Avg@3 is 72.52 / 62.34 on AppWorld against GRPO 64.89 / 53.31, 40.00 / 32.50 on SWE-Gym against 28.13 / 25.63, and 76.33 / 73.44 on DBBench against 69.67 / 67.00. It is also ahead of 9B H0 rescue, at 65.65 / 54.83, 29.38 / 26.88, and 70.67 / 67.78.
Rank-1 routing, with no extra executor rollouts, lifts 4B Pass@3 to 68.32, 42.50, and 75.00. That routed 4B beats 9B H0-rescue GRPO on every reported metric. Routed 9B reaches DBBench Pass@3 78.00 and Avg@3 75.67, above GPT-6-Sol, DeepSeek-v4-Flash, and GLM-5.3-Flash on that benchmark (frontier Pass@3 tops out at 76.67, Avg@3 at 72.11). AppWorld and SWE-Gym stay behind the frontier references. GPT-6-Sol scores Pass@3 82.82 and 53.13 there.
With the starting policy frozen, Harness-S (HE only) scores 56.87 Pass@3 on AppWorld 4B, choosing a harness per task. Meta-Harness uses one Hbest for every test task and scores 48.09. The trained policy back on H0, at 67.56, is above that frozen result and above WHALE's 63.36 on its own Hbest.
On AppWorld 4B, the effective-group rate (retained groups that contain both successes and failures) is 79.38% for Harness-S across the run, 70.94% for H0 rescue, and 64.69% for plain GRPO. Those groups account for 80.88% of retained policy tokens under Harness-S, against 72.68% for H0 rescue. On the same validation set, final H0 success is 63.33%, 54.00%, and 48.00% for the three. Both fixed harness mixtures finish that validation curve below plain GRPO.
Ablations hold the rescue rollout budget fixed on AppWorld 4B. Random siblings finish at Pass@3 / Avg@3 62.98 / 50.76. A rule router with no profiles, ranked by success on similar validation tasks, finishes at 66.41 / 52.93. Two rollouts on each of four harnesses finish at 65.65 / 55.60. Top-8 with frozen profiles finishes at 64.50 / 52.16. Top-8 with profile re-anchoring finishes at 67.56 / 56.87, with higher rescue success in all four training stages and Avg@3 up 4.71 points. Rescue success is the share of rescue-triggering tasks with at least one successful rollout. Under that final policy, one sibling repairs 11 to 15 base-harness failures; nine analyzed siblings repair 29 in union. The union gives each sibling one execution. The router has to choose before seeing those outcomes.
Behavior under H0 moves with the rescues. The percentages are the share of full trajectories that show each action. On AppWorld, schema lookup before the first API call is 22.9% for the base policy, 38.2% for GRPO, and 67.6% for Harness-S. Repeated execute-code drops from 30.1% under GRPO to 9.8%. On SWE-Gym, switching to assertions after blocked tests is 48.5% for Harness-S and 13.3% for GRPO. Two held-out cases line up with training rescues. On a set-intersection task, harness H2 at update 10 kept only songs present on both sides. GRPO took the union on a new instance and failed. The trained policy applied the intersection under H0. On account recovery, H5 at update 3 read a code, reset a Venmo password, and checked login. The trained policy replayed that sequence on an Amazon instance. GRPO read the inbox and left the password unchanged.
Router rank predicts execution before and after training. Pass@3 and Avg@3 fall from Rank-1 to Rank-3. The share of tasks for which H0 itself is Rank-1 rises from 1.53% to 22.52%. The Rank-1 margin over H0 shrinks from 14.21 to 0.76 points on Pass@3, and from 13.49 to 4.58 points on Avg@3. After training, H0 beats Rank-2 and Rank-3.
The same 4B policy, with no further updates, also leads plain GRPO under harnesses it was not trained to mix. Pass@3 gains on AppWorld are 7.25, 9.16, and 4.96 points under DeepSeek Harness, Claude Code, and OpenClaw. On SWE-Gym the gains are 14.37, 12.50, and 10.62. Absolute scores on the two external harnesses stay below the score on DeepSeek Harness.
Harness evolution does not have to share an update loop with the policy. Freeze a complementary pool, spend rescue rollouts only where every H0 attempt failed, and evaluate back on H0. The 4B policy on the original harness then beats a 9B H0-rescue run of the same optimizer. Extra base-harness sampling does not account for the gap. H0 rescue already draws new H0 trajectories on the same all-fail trigger.
A deployment can ship the trained policy and leave the execution system alone. Rank-1 is there if consistency across attempts matters: same rollout budget, with 4.58 Avg@3 points still attached. The main results do not show that this pool drops onto a different policy. That reuse question is left open.
Against frontier systems the gain has a clear edge. Routed 9B passes the frontier references on DBBench and stays behind on AppWorld and SWE-Gym. DBBench is the benchmark most sensitive to query and check habits. It is a weak basis for saying a small open policy has caught closed frontier agents.
Transfer is partial. After training, Rank-1 leads H0 by 0.76 Pass@3 points and 4.58 Avg@3 points. Most newly solved tasks sit in the policy. Consistency across attempts still leans on per-task harness choice. Gains over GRPO survive on Claude Code and OpenClaw, but absolute accuracy falls below DeepSeek Harness. The interface still decides whether the learned habits finish the task.
The two transfer cases are single held-out instances, one test and one validation, each linked to a rescue that entered one policy update. There is no rate for how often a rescued behavior is retained.
The router never sees the outcome of the task it is routing. Nine siblings cover 29 base failures in union and 11 to 15 each, and that gap is selection error. Rescue success also fails to rank the final test scores: the rule router beats two-rollout Top-4 on Pass@3 and loses on Avg@3.
The study uses two Qwen3.5 sizes, three benchmarks, and three runs of three rollouts. The pool is evolved against the starting policy. Training refreshes profiles and does not edit harness code. WHALE, HarnessForge, and Meta-Harness report one Hbest for all test tasks, while Harness-S Hrank1 chooses per task. Comparisons that put every method back on H0 are the clean ones. The main text gives no absolute GPU hours. It says full-pipeline GPU time and external-model tokens sit in the range of the joint-adaptation baselines, and that on AppWorld 4B the method uses fewer rollouts and less training GPU time than H0 rescue.
Still open: how long the transfer lasts, whether a pool can be reused across policies, how independently written harnesses would pass task state across interfaces, and whether more repair diversity is worth the calibration cost. The supported claim is narrower. On one execution foundation, complementary siblings can write some execution habits into a small policy by rescuing all-fail groups.