Mitigating Context Interference for Reliable and Efficient Search Agents
Boyang Xue, Bin Wu, Shuofei Qiao, Sheng Wang, Rui Wang, Yiming Du, Hongru Wang, Jeff Z. Pan, Emine Yilmaz, Kam-Fai Wong, Aldo Lipani
cs.CL
2026-08-11
Tracing search-agent interference mostly to the latest retrieved docs, they distill a refiner from GPT-4 and bake it into GRPO; the 7B CRRL hits 36.6 avg EM with fewer retrievals and shorter context.
A multi-turn search agent is an LLM app that breaks a question into steps and alternates retrieving with reasoning until it can answer. Frameworks like IRCoT dump the documents retrieved each round straight into the context. After a few rounds the context is long and mostly irrelevant to the current sub-question, and the LLM gets distracted: wrong answers, or extra retrieval rounds. The authors call this context interference and ask three things: which part of the context causes it, whether it can be cleaned before generation, and whether folding that cleanup into training helps further.
Diagnose first, then treat.
For diagnosis they ablate by category, building IRCoT variants. IRCoT-o drops previous retrieved documents, IRCoT-oq also drops previous queries, IRCoT-oqp drops previous reasoning too. The result is counterintuitive. Removing prior context does not hurt; IRCoT-o actually beats IRCoT. But the gap between recall rate and recall accuracy stays wide, so prior rounds are not the main culprit. The real source is the freshly retrieved document from the current round: it is the largest chunk and the one closest to the step being generated, so its noise hits hardest.
For treatment they distill. A teacher GPT-4, given the previous query and the current retrieved document, extracts the critical information. An entailment model then verifies that the extract is fully supported by the source and adds no new knowledge, guarding against teacher hallucination. These document-to-extract pairs SFT a small base model into a context refiner F. At inference, each round runs F to compress retrieved documents into key points before the LLM generates.
Step three is CRRL (Context-Refined RL). The refiner is wired into GRPO, trajectories hold only reasoning steps and refined documents (raw documents never enter), and advantages come from group-relative rewards.
Seven QA datasets (single-hop: NQ, TriviaQA, PopQA; multi-hop: HotpotQA, 2Wiki, MuSiQue, Bamboogle), 7B model, EM for accuracy and ART for retrieval count.
| Method | Avg EM | Avg retrievals |
| IRCoT (baseline) | 27.5 | 2.6 |
| Refine only (no RL) | 32.2 | 1.2 |
| Search-GRPO | 34.6 | 2.1 |
| Search-o1 | 36.2 | 1.9 |
| CRRL (this work) | 36.6 | 1.7 |
Efficiency gains are larger. CRRL's average context is 0.7k tokens versus IRCoT's 2.3k, and per-question inference time is 16.9s versus 22.4s. To be honest, against the strongest RL baseline Search-o1, CRRL's average EM is only 0.4 higher; the accuracy gain is modest and the real win is shorter context and fewer retrievals. On the hard MuSiQue set CRRL (13.2) even trails Search-o1 (13.4).
The diagnosis itself is reusable. In a long-context agent the noise does not get worse the older it is; the latest input is what hurts. That is a different axis from lost-in-the-middle, which says information in the middle of a context gets ignored, and together they give concrete guidance for designing context management. On the engineering side, a distilled small model as refiner is far cheaper than calling GPT-4 every round to compress (GPT-Refine: 33.2/1.2), with comparable results. It turns think-before-you-read-the-sources into a deployable middle layer.
The authors state two: they only validate search agents, while tool-use and planning agents suffer the same interference but are not covered; and the refiner is currently an auxiliary module rather than integrated into the agent's end-to-end training.
One more from reading it: CRRL's gain over Search-o1 is small and concentrated on multi-hop tasks; distillation depends on teacher quality and the paper does not discuss what happens when the teacher extracts wrongly. And the claim that the latest document is the main source of interference was found under IRCoT's specific retrieval setup. Whether it holds with a different retriever or a much longer context window is untested.