HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution
Wen Jiang, Mingmin Chu, Yimeng Tian, Qianxin Zhang, Haofei Yang, Rui Yang, Yang Liu, Tao Lv, Fangming Li
cs.LG, cs.AI
2026-09-01
HarnessEvolve aligns failures to verified reference traces, then gates leakage and regression. CloudCoreNetwork-QA with Qwen: 86.9% vs GEPA 65.3% and Base 43.4%.
Self-evolving agents want environmental feedback to rewrite their harness: prompts, skills, tools, and execution logic, with less hand-written workflow. Most methods only see a terminal pass or fail. In long-horizon tasks later mistakes are often downstream of an earlier one, so credit assignment is mush. Unguarded updates hard-code answers and pile task-specific examples; training scores inflate and old competence decays.
Huawei's ICT AI Competence Center presents HarnessEvolve as a systems fix for those three failure modes: credit assignment, shortcut learning, and catastrophic forgetting. The execution agent freezes the underlying LLM and only the harness evolves. Evaluation, optimization, and gating sit in separate modules, so the executor cannot grade itself.
The training set has ground-truth answers. Before the loop, the executor is given the question and the answer for up to 5 attempts and produces a reference trajectory of how the task should have been solved. The evaluator checks that the trajectory actually called tools and reasoned, and rejects reciting the answer. Verified traces go into a global cache.
On each mini-batch of 40: execute; mark failures; align each failure against its reference to the first diverging action; cluster by error type, keeping singleton long-tail clusters; then edit the harness. Missing references fall back to single-trajectory analysis.
Two gates sit on the update. The quality gate is an LLM judge for data leakage (score threshold 0.8) and newly injected in-context examples (cap 5); rejects go back for revision, at most 3 times. The performance gate requires no drop on the current batch and at most 0.025 degradation on the last 2 batches; accepted snapshots enter a pool and become the live agent.
At epoch end the validation set picks the best snapshot in the pool. Ten consecutive performance-gate rejections end the epoch early; five epochs without validation gain stop the run. Cap is 20 epochs.
Baselines are GEPA (prompts), ACE (context), and SkillOpt (skill.md only), same initial harness. In-house data runs on LAMAgent; open-source data on OpenClaw. Models: domain-finetuned Qwen3.6-27B and off-the-shelf DeepSeek-V4-Flash.
On in-house CloudCoreNetwork-QA with Qwen: Base 43.4%, GEPA 65.3%, ACE 59.3%, SkillOpt 61.9%, HarnessEvolve 86.9%, 21.6 points over the strongest baseline. DeepSeek on the same set goes from 47.5% to 85.9%. On Wireless-QA, DeepSeek hits 92.8% versus ACE at 90.1%.
Open-source, DeepSeek plus OpenClaw: SearchQA 92.9% (Base 86.5%, ACE 90.0%); OfficeQA 70.9% (Base 62.8%, ACE 68.9%); SpreadsheetBench 76.4% (Base 44.3%, SkillOpt 74.6%).
Skills evolved on OpenClaw transfer to Hermes, OpenCode, LAMAgent, and DeepSeek Harness with flat or better accuracy. Hermes on SearchQA is 95.0% versus 95.0%: no gain.
Ablation on CloudCore with Qwen: drop reference trajectories and accuracy falls to 57.8%; drop clustering, 68.6%; drop the quality gate, 80.1%. The reference path is the largest piece.
This is engineering guardrails for agents that rewrite their own prompts and skills. If a labeled eval set already exists, aligning against a successful trace beats undifferentiated reflection. The quality gate targets a failure everyone has seen: stuffing items and answers into the harness.
It is incremental systems work. The method edits the whole harness; each baseline touches one slice. How much of the gap is "edit more files" versus diagnosis and gating is not isolated. There is no cross-method control that edits the full harness without reference trajectories.
Reference traces need ground-truth answers, so unlabeled live traffic cannot use this diagnosis. The two in-house sets are private, so the 21.6-point headline cannot be checked outside Huawei. The quality gate is itself an LLM judge. The performance gate watches training batches; validation only picks snapshots at epoch end, so train-distribution overfitting is still possible. Token and wall-clock cost of evolution are unreported. Cross-framework transfer is a wash on the already high Hermes plus SearchQA setting.