Harness Continual Learning: Continual Adaptation Beyond Model Parameters
Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, Yang Gao
cs.LG, cs.AI
2026-08-19
With the model frozen, prompt and memory edits still cause forgetting. HCL gates harness commits; Stability-HCL hits 61.74% on ALFWorld versus 47.12% static.
Continual learning used to mean parameter updates: train on a new task, watch the old one collapse. That is catastrophic forgetting. Many agents now leave the foundation model frozen and edit the surrounding shell instead: prompts, memories, tool specs, routing rules. The paper calls this shell a harness. It decides how the model reads an input, what it retrieves, which tools it calls, and in what order.
Editing that shell can still break old behavior. A memory write changes the evidence an earlier query retrieves. A skill revision changes tool use. A routing edit can snap a workflow that used to finish. The weights never move, yet a correct answer, a valid tool call, or a successful environment trajectory can fail. That is harness-level forgetting: the stability-plasticity trade-off, moved from parameters to runtime state.
Harness engineering usually optimizes a prompt or workflow for the current objective. It rarely asks whether yesterday's tasks still work after the commit. HCL treats the whole mutable shell as the continual-learning state, and makes "do not drop what already worked" a condition of deployment.
The deployed harness is four jointly versioned parts.
Updates follow guarded harness evolution: proposal is separate from commit. A Continual Optimizer reads post-execution feedback and revises components in a fixed order, up to K alternatives per component, swapping only that component. A Continual Evaluator commits only if three gates pass:
Anchors are evaluation-only; the Optimizer never sees them. If nothing passes, the live harness stays.
Experiments span three regimes. ALFWorld uses frozen Qwen3.5-9B on six household task types in sequence. Minecraft uses Qwen3.6-27B on a 50-task curriculum. Textual reasoning uses DeepSeek-V4-Flash on MuSiQue, ProofWriter, GSM8K, then HotpotQA. Multimodal perception uses Qwen3.6-27B on detection, captioning, grounding, then VQAv2. Baselines include a static harness, RAG, MemP, and MemRL, plus DGG on the multimodal stream.
ALFWorld reports 134 official eval episodes. The static harness finishes at 47.12% average. RAG reaches 55.56% with 1.74 forgetting, the lowest among adaptive methods, but retrieval cannot revise procedures. Stability-HCL hits 61.74% with 2.64 forgetting. Plasticity-HCL hits 62.98% with 10.94 forgetting, and Two-object rises from 58.80% static to 100%.
| Method | Final avg ↑ | Avg. forgetting ↓ |
| Static Harness | 47.12 | – |
| RAG | 55.56 | 1.74 |
| MemP / MemRL | 53.15 / 51.51 | 5.18 / 5.64 |
| Stability-HCL | 61.74 | 2.64 |
| Plasticity-HCL | 62.98 | 10.94 |
In Minecraft the static harness plateaus at task 15; HCL finishes all 50, using 83 environment actions versus 88 for MemRL and 91 for MemP. On the text stream, zero-shot averages 45.50. Stability-HCL reaches 52.20 with 0.00 forgetting. Plasticity-HCL reaches 64.70 with 0.07 forgetting, and GSM8K jumps from 49.40 to 92.00. On multimodal, Stability-HCL averages 68.92 with 0.22 forgetting, against 39.40 zero-shot and 42.73 for DGG. Detection moves from 4.27 to 65.34, grounding from 43.00 to 91.60. VQAv2 is the exception: zero-shot 84.87 still beats Stability's 79.33, because the frozen model already answers image questions well.
Sweeping B through 0, 1, 3, and infinity, forgetting rises from 0.39 to 3.45. Peak final average is 63.46 at B = 1; unconstrained B lands at 60.13. More plasticity does not produce a stronger final harness. Ablating memory updates raises forgetting to 0.83 and drops the average to 62.28, versus 63.41 full.
Most teams already freeze the model and edit prompts and memory. This paper treats that practice as continual learning and gives a knob, the historical-loss budget. An internal agent stack can copy the commit gate without standing up parameter-level continual learning.
It is a framework plus a protocol, not a drop-in product. Optimizer and Evaluator both call the frozen model repeatedly, and anchor replay costs tokens. Open issues named by the authors: cheaper retention checks, harness-content consolidation, and longer interaction streams.
Even B = 0 leaves 0.39 test forgetting, because the gate protects a finite anchor set while forgetting is scored on held-out historical tests. MemP and MemRL are reimplemented inside this harness, not run from official repos. The multimodal ablation uses Qwen3.5-4B, a different backbone from the 27B main run. Minecraft reports curriculum completion and action counts, not per-category forgetting. On the text stream, MuSiQue falls below the 35.00 zero-shot score, to 27.60 under Stability-HCL; harness updates can hurt multi-hop retrieval, and the paper does not isolate why. Task order is fixed; there is no shuffle robustness check.