When to search, where to look, when to stop: six capabilities of post-trained LLM search

From Search to Decision: How Post-Training Shapes Information-Seeking Capabilities in Large Language Models

Haohang Huang, Xuan Lu, Mingjin Kuai, Yuxuan Zhang, Wei Ji, Haiyang Zhang

cs.IR

2026-11-10

A survey frames LLM information seeking as six capabilities, compares post-training methods along four axes, and maps where behavioral evidence is still thin.

What problem this solves

Retrieval-augmented LLMs have moved from fixed pipelines to systems that make their own calls: whether to search, what to query, which source to try, whether evidence is trustworthy, when to stop. Post-training methods such as Search-R1, CoBRA, and QREF each improve one of these decisions, but they differ in tasks, supervision signals, and evaluation setups, so cross-paper comparison is nearly impossible. Prior surveys organize the space by architecture, retrieval workflow, or optimization technique. This one, from a team at Xi'an Jiaotong-Liverpool University, Shanghai Jiaotong University, Nanjing University, and UBC, organizes it by the decisions a model must learn to make, and asks for each: what signal trains it, which components change, and how strong the evidence is.

Method

The framework has two layers. The first is six capabilities, defined by the decision each governs rather than by pipeline stage:

The second layer is four training dimensions: learning signals (demonstrations, outcome rewards, behavior-specific rewards, preferences), optimizers (SFT, DPO, RL, hybrids, teacher-guided variants), update loci (a specialized component versus the whole policy; parameters versus external state), and credit assignment (how a final-answer reward propagates to intermediate steps).

The payoff is contrast. Search-R1 optimizes search and reasoning jointly from final-answer correctness; CoBRA uses the reward margin between with-tool and without-tool rollouts to target the retrieval decision alone. TRACE estimates turn-level credit from state-value changes under a frozen reference model; CW-GRPO reweights outcome advantages by judged per-round contributions. Different signals train different decisions, and downstream scores alone do not expose those differences.

Results

The survey runs no unified experiments and reports no new benchmark numbers. What it does is sort the scattered evidence by capability. The verdict: all six capabilities show measurable behavioral gains, but the evidence boundary varies widely.

CapabilityEstablished evidenceUntested boundary
Epistemic awarenessRouting accuracy and costCalibrated ex-ante utility
Query formulationRetrieval gain and coverage from rewritingSemantic intent preservation
ExplorationRecall and diversity in multi-tool searchDiscovering necessary evidence; tools fixed
Evidence evaluationSupport and sufficiency metricsSource reliability judgment
Evidence integrationGrounded QA and hallucination metricsCausal evidence reliance
Adaptive search controlAccuracy and efficiency of iterative searchFeedback-sensitive next actions

Two cross-paper comparisons stand out. Under controlled training conditions, Zhao et al. find that corpus coverage substantially shifts performance and method rankings, and that plain outcome rewards remain competitive against more elaborate credit assignment. On generalization, AnySearch holds up under unseen budgets, CoBRA passes out-of-distribution tool-use questions, QREF transfers across agent backbones, and TRACE moves from closed retrieval to open-web benchmarks. But Yuan et al. show that query strategies learned for one retriever lose effectiveness on a different backend.

Why it matters

For teams building search agents or deep-research products, this is a selection map: pick the decision you want to change, and the survey points to the signal class, the component to update, and the metric to check. Its three evaluation principles (capability-specific measures alongside end-task scores, controlled interventions that isolate one decision, explicit generalization settings) work directly as an internal evaluation checklist. For researchers, the three open problems are ready-made directions: estimating retrieval utility before retrieval, coordinating capabilities under conflicting information, and adapting to unfamiliar retrieval environments.

Expect a survey, not a new method. Its value is the organization, plus one uncomfortable finding: the payoff from fine-grained credit assignment is not robust in controlled comparisons.

Limitations

The authors are explicit that evidence scope varies by capability: source reliability, causal evidence reliance, and semantic intent preservation remain untouched, and exploration studies almost all fix the tool set. The capability boundaries overlap, which the paper waves off with "interrelated"; exploration versus search control is genuinely blurry. As a survey, its cross-paper comparisons inherit each paper's own training corpora and rollout protocols, and the controlled comparison it leans on (Zhao et al.) covers only a few settings. The proposed evaluation protocol is advisory and is not demonstrated end to end on a case study.

Terms

Source

What people are saying

Related papers

All paper explainers