BRIDGE: bilevel RL jointly trains search agent LLM and retriever, boosting multi-hop QA by 9.6 EM

_reachsumit · x · 2026-09-30

A new arXiv paper introduces BRIDGE, a retrieval-credit-aware bilevel optimization method for agentic reinforcement learning.

Motivation: Existing ARL methods optimize only LLM tokens and treat retrieved evidence as environment observations, creating an information-credit gap where retriever failures are blamed on the LLM.
Key finding: Learning order matters—adapting the retriever before optimizing the policy yields larger reward gains than the reverse.
Method: BRIDGE formulates retrieval-augmented agentic RL as bilevel optimization and solves it with a memory-efficient first-order method grounded in loss-landscape analysis.
Results: Across seven open-domain QA benchmarks, BRIDGE achieves the best average accuracy with 3B and 7B backbones, improving multi-hop average over the strongest baseline by 9.6 and 3.4 EM respectively, plus top results on medical QA. Code is open-sourced.

More from coding & agent