CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting
Zixi Zhu, Jiayuan Su, Jian Zhang, Yu Lin, Hongwei Wang
cs.AI
2026-08-21
CAS truncates retrieval with APS and down-weights low-confidence GRPO trajectories via ACI. On Qwen3-8B it reaches 0.464 mean EM versus 0.446 for Search-R2.
Search agents trained with RL interleave thinking and retrieval, then break in two places. Fixed Top-K either drops evidence or dumps noise into context. Sparse binary rewards make the policy more confident as training goes on, so hallucinated answers and extra tool calls rise together. Researchers at Zhejiang University's ZJU-UIUC Institute and Tencent plug conformal prediction (a distribution-free uncertainty method with finite-sample coverage) into both retrieval and training, and call the result Conformalized Agentic Search (CAS).
The bet is not a stronger search policy. It is a statistical gate on the retrieved set and on which trajectories get to update the policy.
On retrieval, Adaptive Prediction Set (APS) takes scored candidates, turns scores into softmax probabilities, and truncates once cumulative mass crosses a calibrated threshold q̂APS. Easy queries keep a short set; hard queries keep a longer one, with a 1-α marginal coverage target. Default αAPS is 0.2, returning 3.4 documents on average. The calibration set has 239 queries; DeepSeek-V3.2 splits multi-hop items and judges which documents are gold.
On training, Adaptive Conformal Inference (ACI) handles the fact that the policy is non-stationary, so ordinary i.i.d. conformal prediction does not apply. The nonconformity score is mean negative log-likelihood of tokens inside <answer>. A rolling quantile marks low-confidence trajectories, which are down-weighted by η=0.5 in the GRPO loss. The target error rate ρ defaults to 0.25, and αt is tracked with a 0.95 exponential moving average so a single miss does not blow the set open.
Reward is Exact Match plus a format term. Policies start from Qwen2.5-3B-Instruct and Qwen3-8B, retrieve with E5 over the 2018 Wikipedia dump, sample five GRPO trajectories, and cap search at four rounds. Training mixes NQ and HotpotQA.
Across seven open-domain QA sets, CAS on Qwen3-8B reaches 0.464 mean EM, ahead of Search-R2 at 0.446 and Search-R1 at 0.400. On the 3B backbone the mean is 0.401, +0.065 over Search-R1. The gap is mostly multi-hop: +0.078 vs Search-R1 and +0.032 vs Search-R2 on the 8B model. On single-hop, CAS beats Search-R2 by 0.001.
| Method (Qwen3-8B) | Single-hop | Multi-hop | Overall |
| Search-R1 | 0.496 | 0.329 | 0.400 |
| Search-R2 | 0.540 | 0.375 | 0.446 |
| CAS | 0.541 | 0.407 | 0.464 |
3B ablations: drop ACI and the overall score falls from 0.401 to 0.384, with noisier search counts; drop APS and fall back to fixed top-3, and the score is 0.389. Setting ρ=0.4 flags about 40% of trajectories as low-confidence and the score drops to 0.368. Setting αAPS=0.05 returns 4.8 documents, lifts single-hop to 0.504, and collapses multi-hop to 0.234. The default is a coverage-versus-noise trade-off. More conservative is not uniformly better.
For teams already training Search-R1-style agents, this is a plug-in: keep the backbone, change truncation and trajectory weights. APS replaces a guessed K with a coverage target. ACI reads confidence from answer-span likelihood, which still works when tool calls break an explicit verbalized-confidence scheme. Code is public. Single-hop is essentially tied with Search-R2; multi-hop accuracy and fewer redundant searches are where the method pays.
Almost all numbers are open-domain QA; specialized domains are untested. APS calibration depends on DeepSeek-V3.2 to split queries and judge relevance, so a weak judge can hollow out the coverage claim. Guarantees stop at the final answer, not intermediate steps. Search-R2 is closed-source, so the 3B comparison lacks that baseline. η, ρ, and αAPS were grid-searched; a new backbone or retriever needs recalibration. Coverage still wants the calibration set close to test, and early RL is a large distribution shift. ACI is there to track that shift; the first-step spike in low-confidence mass is evidence that static conformal prediction fails in this setting.