GitHarness: Git Init Your Harness Working Memory for Perpetual User Requirements
Zhibang Yang, Xinke Jiang, Yuxuan Liu, Mingyu Zhang, Zhixin Zhang, Zhengxing Song, Yue Fang, Guohong Qiu, Ruiqing Li, Xu Chu, Junfeng Zhao, Yasha Wang
cs.MA, cs.SE
2026-09-29
GitHarness stores requirements and work as Git versions. A compatible base plus local edits beats native execution in all 30 settings and cuts code tokens 73.6% versus native.
Users rarely freeze the spec before a long-horizon agent has already searched, coded, and drafted. After seeing that work they fill in missing constraints (requirement completion), add a new dimension (requirement elicitation), or revise and withdraw something (requirement shift). The edit usually hits a small piece: one bug, one section of a report, the color of an umbrella.
Retrieval and intent rewriting can say what is wanted now. They do not decide which old evidence, code, or drafts are still valid, so the run is often started over. Trajectory compression and checkpoints assume the goal stays fixed, and stale requirements keep their artifacts in context. Keeping everything pollutes the next step. Restarting rewrites parts that were already right. Someone has to connect the requirement change to a restore point and to what may stay.
GitHarness sits outside the harness. A harness is the scaffold around the model: memory, tools, and workspace. The executor stays fixed, and each harness keeps its own working memory. Every successful turn commits one version, the requirement then in force plus a recoverable workspace checkpoint. Versions form a directed graph. Old checkpoints are immutable. Failed runs are not stored.
New feedback takes two steps.
The system does not predict which files will change. It resumes from a still-compatible state and lets the new requirement bound the edit. The update hint passed downstream is only a hint.
Only the Git Agent is trained. The harness and the task model are a black box, and gradients hit only the Git Agent's output tokens. That interface-level setup is Harness RL. The outcome reward is whether the final artifact passes the task verifier. A per-turn query-tracking reward, from an LLM judge, checks for omitted or invented requirements, a base that reuses as much compatible work as it can, and withdrawn requirements that were actually removed. Samples for the same prompt are updated with GRPO, group-normalized advantages. The tracking weight λ is 0.2.
MTAgentBench turns five benchmarks into seven-turn dialogues: GSM8K, BIRD, BrowseComp-Plus, SWE-bench Verified, and DeepResearch Bench. The ending requirement matches the original task semantically, so the original metrics stay: accuracy, SQL execution accuracy, answer accuracy, resolved rate, and RACE, DeepResearch Bench's report score against its reference reports. Harnesses are TCRAG, StackPlanner, and OpenHands. Backbones are Qwen3-32B and DeepSeek-V4-Flash. Baselines are native continuation, Restart on a clean workspace each turn, U-Fold, and GCC, which also uses Git operations to manage context.
Training-free GitHarness beats Native in all 30 settings, two models times three harnesses times five tasks. It strictly beats the strongest baseline in 27, ties one, and loses two. The losses are SQL on StackPlanner with DeepSeek, 78.0 against U-Fold at 81.0, and SQL on OpenHands with Qwen3-32B, 73.0 against GCC at 77.0. The tie is OpenHands with Qwen3-32B on search, 10.0 each.
| Setting | Metric | GitHarness | Best baseline |
| TCRAG + DeepSeek, search | Accuracy | 56.0 | GCC 42.0 |
| OpenHands + DeepSeek, search | Accuracy | 57.0 | Restart 39.0 |
| StackPlanner + DeepSeek, math | Accuracy | 94.0 | U-Fold 80.0 |
| StackPlanner + DeepSeek, code | Resolved rate | 74.0 | U-Fold 73.0 |
| StackPlanner + Qwen3-32B, SQL | Execution accuracy | 72.0 | Restart 66.0 |
Search on DeepSeek is the wide margin: 14.0 over the best baseline on TCRAG and 18.0 on OpenHands, or 22.0 and 30.0 over Native. Qwen3-32B's raw search scores mostly sit between 1 and 15, and its code scores between 3 and 19.
Tokens are seven-turn StackPlanner runs with DeepSeek-V4-Flash, against Native. SQL drops 35.8% (149.68k to 96.10k), search 54.5% (1438.03k to 653.87k), code 73.6% (4900.13k to 1291.43k), and research 36.7% (2095.92k to 1327.77k). On those last three, GitHarness is both the best score and the cheapest run. Math tokens rise from 43.13k to 59.98k. Short calculations are cheaper to redo than to audit. Estimated code API cost falls from $2.28 to $0.63 per task.
Locking the base to HEAD drops the code resolved rate from 74.0 to 44.0 and raises tokens from 1.29M to 4.51M. Removing INSPECT leaves a resolved rate of 42.0 and 4.22M tokens. Disabling a fresh solve hurts all five tasks, by 10.0 points on SQL and 11.0 on search.
Harness RL trains only a Qwen3-8B Git Agent. The downstream executor stays DeepSeek-V4-Flash. Math moves from 51.0 to 81.0 and SQL from 55.0 to 73.0. On a 200-task validation set, λ = 0 scores 47.0 on query equivalence and 70.0 on the task. λ = 1 raises query equivalence to 53.0 and drops the task score to 65.0.
A fixed skill memory with no evaluation items, retrieved at test time, improves 13 of 15 settings. StackPlanner SQL goes from 78.0 to 83.0. Search, code, and research supplied no trajectories for that memory, yet research RACE still rises by 2.07 to 4.27 across the three harnesses. With two to four tasks interleaved out to 30 turns, math and research both rank first. At 30 turns, math is 84.0 against U-Fold at 78.0, and research RACE is 47.32 against Native at 34.53. From 10 turns to 30, research falls by 2.21.
The Peking University layer does not retrain the executors inside OpenHands, StackPlanner, or TCRAG. An 8B Git Agent on a frozen DeepSeek executor gains 30 points on math and 18 on SQL, so the choice of where to resume can be trained apart from how the task is finished. Cutting code tokens by about three quarters means part of the bill was recomputation of work that should have been dropped.
GCC and GitOfThoughts also commit and branch. They pick history from the current line of reasoning or from failure recovery, and the goal is assumed stable. GitHarness picks the base from how the requirement changed, then keeps, drops, or recomputes. The higher scores come from less rework. The executor did not suddenly get better at the task.
The paper names two holes. Ambiguous or conflicting feedback makes requirement reading and base selection unreliable. Restoring a compatible state still does not stop the underlying agent from touching work that should have stayed put. There is no direct rate of unrelated files changed. Task score and tokens are indirect.
The seven-turn chats are built backward. The source item and its verifier are pinned as the ending, then earlier turns are written as completions, added constraints, and revisions. The ending must stay semantically the original item, and a temporary constraint can be cancelled by a later withdrawal. The test asks whether the original answer still comes out after the requirement wobbles. A user who replaces the goal with a new task that has no gold answer is outside this setup. The chats are not real user logs. Each domain has 100 test items.
The tracking reward comes from an LLM judge. At λ = 1, tracking improves and the task score falls, so the two objectives pull apart, and nothing checks whether the judge and the Git Agent accommodate each other. RL uses 100 GSM8K items and 100 BIRD items. Search, code, and research are out of domain. Under StackPlanner with DeepSeek, the 8B RL agent loses to the untuned DeepSeek Git Agent on all five tasks, 81.0 versus 94.0 on math. The 30-for-30 win over Native is the training-free system. The RL system wins 29 of 30.