AREX: Towards a Recursively Self-Improving Agent for Deep Research
Shuqi Lu, Chaofan Li, Kun Luo, Zhang Zhang, Hui Wang, Hongwang Xiao, Zheng Liu, Lei Xiong, Jiahao Wang, Sen Wang, Xiyan Jiang, Wanli Li, Yuyang Hu, Hongjin Qian, Bingyu Yan, Ziyi Xia, Yingxia Shao, Kang Liu, Zhicheng Dou, Di He, Chaozhuo Li, Qiwei Ye, Zhongyuan Wang, Zheng Liu
cs.AI
2026-07-24
AREX exploits the asymmetry that discovering answers is costly but verifying them is cheap, using an inner research loop and an outer loop that recursively self-improves by confidence; absolute scores are mid-pack, and the real claim is the roughly 23-point self-improvement gain in ablation.
Deep research requires answers that jointly satisfy multiple constraints. Discovering such answers is costly over a large search space, but verifying a candidate often decomposes into tractable per-constraint checks. This discovery-verification asymmetry says a research agent should verify more and use verification to drive the next, more focused round. Simply searching longer fails because early errors persist and directions repeat.
AREX is a bi-level recursive process.
The inner loop executes research actions, integrates evidence, and maintains a trajectory as working research state; when enough evidence accrues it outputs a provisional answer with supporting evidence and an answer-level confidence score.
The outer loop evaluates the result. If confidence clears a threshold it accepts; otherwise it decides whether the trajectory is recoverable (Refine) or should be discarded (Restart). On Refine it preserves verified findings and converts unresolved problems into targeted objectives for the next round.
The confidence score is a compact summary of the evidence-grounded belief state governing termination. The outer loop turns verification into an active control signal that recursively refines state, not a final filter.
Autonomous Context Updating (ACU) is a dedicated tool that compresses the growing interaction history into a compact improvement state, preserving verified evidence and unresolved constraints; the model itself decides when to call it. In the data, 80.3% of cases invoke updatecontext, on average at 25,721 tokens (well under the 128K limit), 66.9% triggered by "revise search strategy," and it reliably preserves unresolved constraints (95.5%), next-step plans (96.4%), and rejected candidates (81.5%). The finish interface externalizes only answer, evidence, and confidence.
Training is multi-stage agentic mid-training: first browse-intensive tool use, then expert reasoning, to avoid interference; key-step supervision, since key steps (first evidence-bearing tool call, rejecting a wrong hypothesis, key context updates) carry 19-29% higher loss than ordinary steps; and step-aware RL with turn-level policy optimization and hierarchical normalization so longer trajectories do not dominate, plus a key-step bonus.
AREX-Base (122B-A10B MoE, 10B active) reports BrowseComp 82.5, GAIA 85.4, xbench-2510 71.0, DeepSearchQA 89.9, WideSearch-en 82.0. These absolute scores are not leading: Kimi-K2.6 is higher on several (BrowseComp 83.2, xbench-2510 90.0, DeepSearchQA 92.5), and MiroThinker-H1 reaches 88.2 on BrowseComp and 88.5 on GAIA. AREX-Turbo (4B dense) scores 70.7 on BrowseComp.
The real claim is not the absolute score but the self-improvement ablation. On BrowseComp, no ACU and no outer loop scores 59.6, plus outer loop 69.8, plus ACU 71.4, full AREX 82.5, a roughly 23-point contribution from self-improvement. Training ablations: progressive training swapped for mixed 77.5; key-step supervision swapped for random replay 74.1; step-aware RL swapped for standard GRPO 79.4.
This reframes deep research from "search more rounds" to "recursively fix yourself with verification signals," and the framework is directly useful for agent teams: confidence gating, Refine and Restart, and ACU are transferable designs. Key-step supervision (concentrating on high-loss decisions) is a useful recipe for agent RL.
Read the absolute scores in context. BrowseComp has historically been hard for frontier models; the 2026 cohort pushes it into the 70-90 range, and AREX's 82.5 is mid-pack among them, so do not sell "BrowseComp 82.5" as SOTA. The real contribution is the 23 points in the ablation. The system runs under a preset maximum rounds and confidence threshold, with no sensitivity analysis on either. Key-step annotation relies on verifiable task structure with reference answers, built offline for training and not used at evaluation; out-of-distribution transfer is unclear. Appendix trajectory self-distillation (plus 4.8) did not make the final recipe, and the authors warn it can inherit or amplify teacher bias.