What is Missing from AI Post-Training AI: An Empirical Analysis
Joy Jia Yin Lim, Xin Huang, Hao Peng, Yaxi Lu, Xin Cong, Zhong Zhang, Maosong Sun, Yankai Lin
cs.AI, cs.CL, cs.LG
2026-08-20
On 1,338 PostTrainBench runs, agents lock strategy before training (2.1% of 3,557 pairs switch). An experience scaffold adds 40.8 HumanEval points and zero revisions.
Frontier LLM agents can already post-train a model end to end: write code, launch jobs, score checkpoints, fix bugs. PostTrainBench made this a standard exam, and agents do raise downstream scores inside a 10-hour budget. The public conversation often treats finishing a training run as if it were doing AI R&D.
This paper splits that into two layers. Execution-level work stays inside a chosen strategy: learning rates, data formats, rewards, checkpoints. Strategy-level work revises the high-level bet as evidence arrives: switch the training paradigm, add or drop a stage, reallocate the remaining budget. The corpus is 1,338 public trajectories across seven benchmarks, four 1.7B to 4B bases, and 20 agent setups. Each run gets one H100 and ten hours.
The strategy is locked before the first line of code. The rest of the budget is local search.
A training experiment counts only if parameters actually update. Script writing, installs, and evals do not. A strategy change between adjacent trainings requires a shift in paradigm, data-source type, or stage structure; everything else is an execution change. An LLM labels from commands and context; the authors review.
They then run three escalating interventions on Qwen3-1.7B-Base, with GSM8K, HumanEval, and AIME 2025 as an easy-to-hard ladder. Autonomous baselines are Claude Code (Opus 4.6 and GLM-5.2) and Codex CLI (GPT-5.2). All interventions sit on Claude Code plus Opus 4.6, three independent 10-hour runs on four A800s. The design is a process of elimination: if lock-in is missing information, a missing person, or missing thought, adding those should move the strategy.
Execution is already competent. Trajectories average 3.82 trainings and 13.80 evals, and every benchmark improves over the base, 10.41% to 23.0% overall. Strategy barely moves.
| Agent | Default | Strategy-change rate |
| Claude Code | Full SFT 80.7% | 4.7% |
| Codex CLI | PEFT 89.6% | 1.6% |
| OpenCode | Full SFT 66.3% | 0.4% |
Of 3,557 adjacent training pairs, 74 (2.1%) from 44 trajectories ever try an alternative. On the same task, Claude defaults to full SFT and Codex to PEFT. The lock is the agent's prior, not the task.
The experience scaffold thickens execution:
| Setting | GSM8K | HumanEval | AIME 2025 |
| Base | 10.84% | 5.48% | 0.00% |
| Official instruct | 88.70% | 66.46% | 33.33% |
| Opus 4.6 autonomous | 64.70% | 22.00% | 3.33% |
| Experience scaffold | 77.30% | 62.80% | 5.56% |
HumanEval jumps from 22.0 to 62.8, a few points short of the official instruct model's 66.46. Dropping the evaluator hurts most (HumanEval 42.60). Strategy advice is adopted zero times. On HumanEval the evaluator recommends switching to RL in 7 of 9 cycles; the main agent writes GRPO scripts, then launches 14 SFT variants and no RL. On AIME it suggests an SFT warmup three times; the agent stays on GRPO-only. Execution suggestions: 8 of 8 adopted. The agent also creates no new skills, even with a create-skill tool in the library.
Human guidance can redirect the opening. Best-run AIME pass@8 goes from 3.33% (baseline) and 6.67% (scaffold) to 13.33%. The agent even inspects the base model, decides the output format already works, and skips the SFT warmup the reviewer asked for. After training starts it collapses back into local loops, mostly about how to resume from an old checkpoint. On AIME the scaffold spends 7.9 times the baseline tokens, about 66.7M tokens for one extra solved problem, inside evaluation variance.
For people building agent systems this is a precise diagnosis. Docs, a better first plan, and extra thinking tokens all fail to reopen a locked strategy. The strategy is plastic only in the short window before the first training run. Once that window closes, the same information, advice, and compute stop working.
The advertised AI-for-AI loop closes at the execution level and stays open at the strategy level. Workflows are locally iterative and globally linear: initial strategy to final checkpoint, almost no turn. Ten hours can be spent efficiently inside the wrong bet. Raising the ceiling means training signals and interaction protocols that make reopening the strategy an explicit, rewarded action. A larger model is the wrong first lever.
Strategy changes are LLM-labeled then human-reviewed. The coarse axes (paradigm, data source, stage) will miss gray-area edits. AIME has 30 problems, so one extra solve is 3.33 points; the paper itself treats pass@8 gaps as inside variance, and the human-guidance score should not be read as a hard win. Interventions sit only on Opus 4.6 plus Claude Code, three runs each. Transfer to other scaffolds is untested. The human steps in once, before training, so this is not a test of ongoing human-in-the-loop. The paper also notes that frequent switching is not automatically better: switches cost real compute. The missing piece is evidence-based comparison of whether a switch is warranted.
Bases stop at 1.7B to 4B. On larger models, strategy search is more expensive; whether lock-in gets worse, or whether execution-level repair gets stronger, is unmeasured.