Latent Programming Horizons in Coding Agents
André Silva, Han Tu, Martin Monperrus
cs.LG, cs.SE
2026-07-06
Linear probes on coding-agent residual streams decode parse, tests, progress and regressions (AUC up to 0.83). Signal stays above chance about 25 steps before the edit hits disk.
A coding agent repairing a real repo spends dozens of steps reading files, editing, running tests, and editing again. Tokens and tool calls are visible. Whether the underlying language model keeps an internal ledger of the current program is not. Prior probes mostly target one-shot function generation, where the whole program sits in context and never changes. In the agent setting the program is only partly observed and mutates at every edit.
Game-playing networks linearly encode the board. Models trained on Karel grid programs encode current and future semantic state. This paper measures whether the same linear readability appears on SWE-Bench, real repositories, multi-step edits, and how far ahead that representation runs.
KTH ran mini-swe-agent v2.2.8 with Qwen3.6-35B-A3B and Laguna-XS.2 on all 500 SWE-Bench-Verified tasks and all 731 SWE-Bench-Pro tasks, up to 10 trajectories per task. That yields 22,714 trajectories, median 52 steps, 79,480 code edits, 22.4 million hidden-state vectors. Both models have residual width 2048. Hidden states are recorded every 5 tokens at layers 1, 11, 21, 31, and 40.
Four binary labels are computed outside the model, by checking out each program version after an edit:
A separate logistic regression is trained per property, per layer, per lookahead k. k=0 reads the current program. k>0 uses the hidden state now to predict the label k steps later. Splits are by task ID, so no task leaks across train and test. A shuffled-label retraining checks that the signal lives in the representation, not in probe capacity.
Current program properties are linearly readable. Semantic labels are strongest. Qwen3.6 reaches AUC 0.83 on full correctness (0.828 at layer 31 on Verified, 0.832 at layer 31 on Pro) and 0.84 on partial correctness (0.841 at layer 21 on Pro). Laguna-XS.2 trails: full correctness around 0.73, regression 0.745 on Pro. Well-formedness on Verified sits near chance (AUC always below 0.60) because the positive rate exceeds 0.92: the models almost always emit compiling Python. Pro mixes in C++ and TypeScript, the positive rate falls to 0.52-0.57, and well-formedness AUC rises to 0.78. Shuffled-label controls sit at 0.50.
Layer-wise the curve is a stable inverted U: weakest at layer 1, peak in the middle, a slight drop at the last layer toward next-token features. Qwen leads Laguna by about 0.10 AUC on the semantic probes. The encoding site is the same; only linear separability differs.
Probes transfer across benchmarks without retraining. Full and partial correctness keep AUC 0.63-0.78 out of distribution against 0.71-0.84 in distribution, a drop of 0.04-0.09. Well-formedness transfer collapses, because the two benchmarks have very different syntactic-failure rates.
Lookahead is the sharper result. On Laguna, full-correctness AUC at k=0 is about 0.77 on Verified and 0.82 on Pro. It falls steeply over the first 25 steps, still about 0.55 and 0.65 at k=25, then plateaus above chance out to k=50 at about 0.52 and 0.60. The paper names this reach the latent programming horizon.
A coding agent's residual stream already holds a linearly readable ledger of the program, and that ledger runs ahead of what has been written to disk. If the representation is causal, one could monitor, halt, or steer along the probe direction before the next patch is emitted. This paper does not show causality, only decodability. For interpretability work, Qwen3.6-35B-A3B is the cleaner probe target: the same layers, roughly 0.10 more AUC.
Decodable is not used. The authors say so: causality would need steering along the probe direction and a measured change in the resulting edit. Median edits per trajectory is 2 (mean 3.5), so "25 steps ahead" is mostly reasoning and file reads, not 25 patches. Well-formedness is badly imbalanced; a low Brier score there is an always-positive predictor matching the base rate. Only two mid-size open-weight models and one harness. Hidden states are subsampled every 5 tokens, and the k=50 sweep drops every trajectory shorter than 50 steps.