Can Computation from Earlier Problems Help LLMs Solve New Ones?
Jipei He, Wenhui Tan, Xiaoyi Yu, Enver Sangineto, Fiorenzo Parascandolo, Rita Cucchiara, Ruihua Song
cs.AI, cs.CL, cs.LG
2026-09-30
STAIR trains 12,288 parameters on a frozen backbone to reread earlier turns' KV cache, lifting later-turn AIME 2025 accuracy by up to 11.67 points over the unmodified model.
Chat products routinely push several independent problems through one session. To the user, earlier turns are just text sitting in context. To the model, generating that text also left computation behind: attention keys and values from earlier responses stay readable to the new problem's queries.
The effect is unstable. On MATH-500, Qwen3-4B Instruct drops from 95.00% Avg@4 without history to 93.75% at turn 4 with it. On GPQA-Diamond, Qwen3.5-4B climbs from 63.64% to 75.25%. Paired answer tracking on MATH-500 shows both directions at turn 4: 35 problems flip from wrong to correct, 60 flip the other way. Leftover computation helps and hurts at once, and the model has no handle on which it gets.
The analysis half replays a frozen backbone under a balanced crossing of 128 current problems with 32 three-turn histories per group, splitting the history-induced shift in each problem's representation into problem effects, history effects, and an interaction term. Current queries genuinely read the past: at layer 3, about 39% of attention mass lands on earlier assistant responses, and about 12% remains at layer 19. The interaction term carries only 0.69% of total displacement energy at layer 19, yet its geometry repeats. Nearest-neighbor sets among problems overlap 48.06% across unrelated histories, against 11.73% for a shuffled-identity control. Something problem-specific and recurring sits inside the history response, which makes a learned readout plausible.
STAIR itself runs in four steps:
The differential readout is the load-bearing choice. The model learns only what the reflection changes, which keeps it close to Native predictions; a KL term against the Native teacher (coefficient 1) reinforces that. Only the reflection normals train, 12,288 parameters for every model. Training data is each backbone's own responses on DAPO-Math-17k, wrong answers included, replayed as four-turn teacher-forced sessions for one epoch.
T2-T4 mean Avg@4, all conditions with history, Native as the reference:
| Setup | Native | STAIR | Delta |
| Qwen3.5-4B, AIME 2025 | 40.56 | 52.22 | +11.67 |
| Qwen3.5-9B, AIME 2025 | 43.33 | 49.44 | +6.11 |
| Qwen3-4B Instruct, AIME 2025 | 34.44 | 38.06 | +3.61 |
| Qwen3.5-4B, MATH-500 | 94.78 | 96.27 | +1.49 |
| Qwen3.5-9B, GPQA-Diamond | 79.88 | 75.93 | -3.95 |
AMC23 adds 6.37 points for Qwen3.5-4B (68.14 to 74.51). On MATH-500 both Qwen3.5 models pass the no-history Vanilla reference (96.27 vs 96.00; 96.78 vs 96.65). AIME is different: Vanilla stands at 60.83 for the 4B against STAIR's 52.22. STAIR repairs the with-history condition; it usually does not beat clearing context.
A shared-T1 control on 30 AIME problems is the cleanest comparison: Native 21.67%, STAIR 48.33%, Pass@4 up from 56.67% to 80.00%. Ablations from the same setting: random reflection directions 37.22%, the full reflected read without differencing 41.67%, a bank-free controller with the same 12,288 parameters 36.67%, permuted K-V pairings 33.06%. Each component earns its place. Against LoRA on Qwen3-4B Instruct: STAIR gets +3.61 from 12,288 parameters, rank-2 LoRA gets +5.00 from 4.13 million (336x as many), and joint training lands 3.89 above LoRA alone, so the two adaptations stack.
Costs: full-bank reading adds a median 0.81 s and 6.60 GiB of peak memory; the median bank holds about 63.4K tokens, and exact replay to build it costs a median 7.29 s per turn. Truncating the bank to 2K/8K/32K tokens drops accuracy from 48.33% to 35.00/43.33/39.17%.
Accuracy decay across long multi-problem sessions is a real serving problem; agent loops hit it constantly. Existing cache-reuse work such as Prompt Cache and CacheBlend saves prefill compute, while this paper treats leftover computation as an accuracy resource and learns to read it. With a frozen backbone, 12,288 parameters, and compatibility with LoRA, the engineering footprint is close to zero. The mechanistic finding also stands alone: history-induced shifts contain a recurring problem-dependent component, so what to read is learnable structure rather than noise.
The authors list three: reflection directions are trained on math only, and GPQA gains do not transfer (the 9B actually drops); evidence covers four-turn sessions within the Qwen family; the bank grows linearly with session length and selective retention is left undone.
Two more caveats from the numbers. AIME 2025 has 120 responses per turn, so one flipped response moves a turn by 0.83 points; the headline 11.67 lands on the smallest, noisiest benchmark, while MATH-500 moves at most 1.49 points (4B) and under 0.6 for the other two models. The anchor throughout is Native-with-history, so the honest framing is a cheap remedy for a condition you could also just clear, not a win over an empty context. And the 7.29 s per-turn replay plus 6.6 GiB of memory rules out latency-sensitive serving in this form.