LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
Zhixin Zhang, Xinke Jiang, Zhibang Yang, Weixuan Xu, Guohong Qiu, Xu Chu, Junfeng Zhao, Yasha Wang
cs.LG, cs.AI
2026-08-12
LoongReflect casts reflection as a reversible memory-control policy with reflect/backtrack actions, trained via teacher distillation plus GRPO, and beats the strongest baseline by 12.6 multi-hop QA F1.
Long-horizon agents (the kind that plan, call tools, and keep memory) need reflection: judging whether the trajectory is progressing, what evidence is missing, which step is unreliable, and whether to continue, revise, or abandon. Learning good reflection has two hard parts. One is the learning-signal dilemma: reflection's value only shows up at the end, so outcome-based RL gives supervision that is sparse and delayed; give reflection its own intermediate reward and the model games it, producing pointless repeated reflection. The other is the local-global gap: reflection is done from the local context of the current branch, but its usefulness depends on the whole trajectory, which a local reflector cannot see.
LoongReflect (Peking University) casts reflection as a memory-control policy, no longer unconstrained free text. The agent runs on a reversible trajectory tree with two control actions:
Only the active path and compressed memory feed generation; abandoned branches stay in the tree for diagnosis without contaminating later decisions.
Training uses two complementary channels, coordinated by look-ahead:
Trained on HotpotQA and 2WikiMultiHopQA, evaluated on seven QA benchmarks.
| Model | Avg F1 | vs AgenticRAG-R1 |
| Qwen2.5-3B | 46.15 | +12.60 |
| Qwen2.5-7B | 49.21 | +12.61 |
It ranks first on every benchmark. Out-of-domain gains hold too: on 3B, in-domain rises from 38.46 to 52.09 and out-of-domain from 31.59 to 43.77. It also transfers to retrieval-free math: MATH 56.0, GSM8K 82.4, though the gains are small (+1.2 and +1.8).
Ablations show both actions and both channels are necessary: dropping reflect costs 15.31 avg F1, dropping backtrack 13.06; dropping the slow channel 7.04, the fast channel 5.64, look-ahead coordination 4.94. By training stage, SFT alone adds 4.43 and the two-channel RL adds another 11.39, so RL supplies most of the gain. This uses only 600 SFT trajectories and 100 RL outer steps.
For anyone building search or tool-using agents, this is a way to train a model to reflect and turn back, not just prompt it at inference. The reversible trajectory tree lets the model recover from bad branches, which matters for multi-hop QA and long-horizon planning. The modest math transfer suggests the learned reflection skill is mainly for retrieval-grounded long-horizon search.
The authors note: dependence on retrieval quality and the controller's checkpoint judgment, where retrieval noise contaminates verified-state construction and early misdiagnoses propagate downstream; a normalized-exact-match reward under-credits semantic aliases; reversible multi-turn search plus the three-fast/one-slow look-ahead adds inference and optimization cost; and limited experimental scope (two training sets, seven QA benchmarks, two math, single seed). One more caveat: several baselines (Mem1, AEPO, ARPO) are weak in this setup, so part of the +12.6 rides on beating weak baselines, and whether the 3B/7B results extrapolate to larger models is untested.