AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks
Hongjin Qian, Chaofan Li, Kun Luo, Wenqing Wei, Jianlyu Chen, Shuqi Lu, Yuyang Hu, Hongwang Xiao, Hui Wang, Chaozhuo Li, Qiwei Ye, Zhicheng Dou, Defu Lian, Zheng Liu
cs.AI
2026-09-30
BAAI fine-tunes Qwen3.8-27B on long-horizon improvement trajectories from ML and coding tasks; it reaches 81.8 on MLE-bench Lite and the skill transfers to deep research.
Agent systems run on a loop of trying, measuring, and revising, but in most of them the loop lives in the scaffold. The harness decides when to retry and which version to keep; the model just produces one attempt at a time. AREX-2, from BAAI, moves the loop into the weights: give the model a bigger round budget and it should produce a better solution by its own judgment. The paper calls this self-improvement, test-time scaling within a single task.
The bottleneck is training data. Standard agent data is built one attempt at a time: pose a task, generate a solution, keep it if the verifier passes. The failed attempts, the feedback they earned, and the revisions that followed are all thrown away. A trajectory where an agent grinds one problem for hours and climbs far above its baseline is nearly absent from training data. A model that has never watched a solution being improved will not improvise the skill at test time.
The hypothesis: judging where a solution falls short, deciding what to change, and continuing after a failed round form a domain-agnostic meta-skill, best learned where supervision is easy. The paper splits it into reflection, which sets the mean gain per productive round, and long-horizon execution, which sets how many rounds stay productive. Total improvement is roughly the product of the two, so a larger budget helps only up to that horizon.
Data construction has three steps:
The training loss is the sharpest design choice. The whole trajectory stays in context, but loss lands only on decisions that move the solution forward, such as diagnosing a failure, repairing it, switching strategy, or submitting an improvement. Polling loops and near-duplicate turns receive none. The model learns what to do after a failure; the failure itself is only context.
AREX-2 is Qwen3.8-27B fine-tuned on these trajectories plus the unchanged deep-research data from the earlier AREX recipe. No new trajectory is a search task, so any gain on deep-research benchmarks counts as transfer.
| Benchmark | AREX-2 (27B) | Comparison | Gap |
| MLE-bench Lite (Any Medal) | 81.8 | Naive-N0.5-Flash 73.7; GPT-5.6 Sol 72.7 | +8.1; +9.1, best in table |
| Frontier-CS | 70.7 | Kimi-K2.7-Code 54.7; GPT-5.6 Sol 76.4 | +16.0, best open-weight; 5.7 behind best closed |
| BrowseComp | 84.0 | AREX (122B) 82.5; GPT-5.6 Sol 90.4 | beats old AREX; trails frontier |
| GAIA | 92.2 | AREX (122B) 85.4 | +6.8 |
| DeepSearchQA | 93.8 | Kimi-K3 95.0; Claude Fable 5 94.2 | third in table |
The process curves are more persuasive than the totals. On Frontier-CS with a five-hour budget, AREX-2 goes from 54.4 at one hour to 65.9 at two and 70.7 at five, still adding 2.2 points in the final hour; DeepSeek-V4-Pro stalls at 44.7 after two. On BrowseComp, with no correctness signal at any point, accuracy climbs from 64.8 at 47 turns to 84.0 at 143. Run under the same process, AREX (122B) gains about one third as much per turn; AREX-2 passes its final accuracy with under half the turns, at under a quarter of the parameters.
The stage-wise ablation on MLE-bench Lite: the base model scores 28.8; general skills lift it to 41.2; task-specific skills plus more rounds reach 68.2; the trained model reaches 81.8. With identical skills and rounds, training adds 13.6 points, and the trained model still gains 6.0 from extra rounds.
This is a route to self-improving agents that skips RL: use verifiable domains as a gym, record sustained improvement, and fine-tune on it. A 27B model beating GPT-5.6 on MLE-bench Lite says the recipe is strong in its source domains; beating the previous AREX models on all four deep-research benchmarks says it generalizes. The comparison tables run up to 2.8T parameters, and 27B holds its place among them.
The ablation also settles a practical question. Skills alone take the base model from 28.8 to 68.2, so a large slice of MLE-bench performance is operational knowledge sitting in context rather than in the weights. If you need points quickly, write the skill documents; the next 13.6 points are what training buys.
The paper concedes gaps on the strongest BrowseComp and HLE results: Kimi-K3 sits at 91.2 on BrowseComp, and Claude Fable 5 reports 64.5 on HLE, though on the full set while AREX-2's 52.6 is the text-only subset.
Other points to weigh. Training is pure imitation, capped by what the teacher agent demonstrated, with no mechanism to exceed it. The transfer claim has a premise: the model already carries AREX's deep-research data, so this is improvement without new data in a familiar domain, not zero-shot crossing. Several baselines were reproduced by the authors themselves (marked \ in the tables), and the MLE-Lite run follows the OpenMLE protocol with skills in context, so cross-paper comparisons need care. The training domains are two; whether domains without automatic scores benefit the same way is untested.