Rehearse: Stepping Back from the Confidence Cliff in Self-Improving Autoresearch
Jiazhen Ji, Shouhong Ding
cs.AI
2026-07-30
Late in autoresearch loops, an LLM judge's accuracy drops from 82.8% to 56.9% yet it keeps deciding. Rehearse's focused outcome memory holds 83.5% and beats baselines on three loops.
Autoresearch loops are a popular class of ML agent: it rewrites training code, runs a full training job for each change, and keeps the changes that move the metric. AutoSOTA and AIDE both work this way.
The authors first pull a pattern out of public AutoSOTA logs: the share of helpful modifications drops from 70% in the first two iterations to 43% by iteration 6+, and even the helpful ones shrink from a 3.6% mean gain to 0.3%. Late in a run, most experiments are executed and then reverted.
The sharper finding is that the agent's judgment degrades too. Before spending a training run, it has to decide whether a proposed change is worth running. That pre-execution judgment gets worse as successes pile up, and the agent does not notice: it keeps deciding just as confidently. They call this the confidence cliff.
Rehearse turns the one-idea-then-run cycle into Propose-Predict-Execute:
A few design choices cut against intuition and deserve to be stated on their own. Focused retrieval (only similar past attempts) beats dumping the full history: cliff selective accuracy rises from 70.8% to 83.5%. Their framing is that structuring a dump does not recover it. You cannot hand the judge an entire trace and hope it sorts things out; you have to pre-filter to relevant precedents.
Binary outcomes only. Recording why an attempt failed lowers accuracy from 83.5% to 80.6%, because failure reasons are mostly noise. Strict consensus (both orders) compresses run-to-run spread from 5.1 to 1.7 points. The motivation is plain: the waste is the execution, so memory has to be read before running.
First, the single-turn benchmark (39 paper-derived AutoSOTA tasks, 366 pairs; nbaseline at least 3 is the cliff bucket):
| Memory config | Overall sel. acc. | Cliff cov. | Cliff sel. acc. |
| No memory | 77.6 | 85 | 56.9 |
| Full-history dump | 82.0 | 80 | 70.8 |
| Self-reflection buffer | 81.0 | 90 | 74.1 |
| LLM summary of history | 81.9 | 78 | 80.9 |
| Rehearse (focused retrieval) | 82.3 | 77 | 83.5 |
Rehearse's overall accuracy is only slightly above the baseline (82.3 vs 77.6). The gap opens in the cliff bucket: no-memory judges at 56.9% yet still commits, while Rehearse judges on fewer calls (coverage 77) and gets 83.5% right.
Then the live multi-turn runs (4,000 budgeted training runs total, five seeds each):
| Loop | Metric | Vanilla | Rehearse |
| nanochat | valbpb reduction | 7.1% | 10.7% |
| CIFAR-10 | accuracy gain | 2.10% | 2.85% |
| ETTh1 | MSE reduction | 40.1% | 54.0% |
On nanochat, propose-many plus select without memory already lifts the endpoint from 7.1% to 8.4%; the store adds the rest to 10.7%. At experiment 30, Rehearse is at 8.9% against vanilla's 3.3%. ETTh1's seed-to-seed spread drops from ±11.2 to ±3.0. Rehearse reaches vanilla's 100-experiment endpoint in 54 (CIFAR) and 63 (ETTh1) experiments, about 46% and 37% fewer. It ends ahead on 9 of 10 additional-loop runs.
For anyone running ML agent loops (AutoSOTA, AIDE), the paper pins down the late-run bottleneck: not a shortage of ideas, but an agent still burning budget on confident failures. Rehearse is a lightweight, deployable skill with no retraining and no model swap, just a propose-compare-with-similar-precedents layer inserted before each run. The cost is extra proposal and judge inference (not counted in their budget), and it can only pick from what the proposer generates.
The authors state three: five seeds and three loops with measurable endpoints, with novelty- or paper-scored systems and larger seed studies out of scope; selection is only from proposer candidates, with other memory forms and candidate counts untested live; the budget counts training runs and excludes proposal and judge inference, so the real cost is higher than the headline.
One more from reading it: the cliff benchmark's nbaseline is observational, not a randomized causal estimate. Later decisions may differ in headroom, modification family, and difficulty, so the cliff is a controlled proxy at run depth. The judge is also a fixed model (deepseek-v4-pro); whether the gains transfer to other judges is untested.