CAFE Improves Search Agents via Co-Evolving Agent and Critic
_reachsumit · x · 2026-08-26
Addressing the limitation of outcome-supervised search agents in localizing intermediate errors, the paper proposes CAFE (Coupled Agent--Feedback Evolution). The framework uses a shared-parameter model alternating between search-agent and critic roles, enabling the agent to request and use corrective feedback mid-trajectory. Combining online RL and offline preference optimization, CAFE allows the agent to learn feedback-guided recovery from its own failures. It outperforms RL-based search agents on average across seven benchmarks and retains gains on six out-of-domain benchmarks.
Related event: Tencent's CAFE Couples Search Agent and Critic for Self-Improvement(2 posts)→
More from Research
- Cornell and Amazon revisit the 16x pretraining compute gap: diffusion LMs need only 3-8x — LucaAmb · 2026-10-16
- AI trained on the fraction of human intelligence that fit the page — r0ck3t23 · 2026-10-16
- Quotient Recurrent Reasoners hit 97.9% on Sudoku-Extreme with 78% fewer loops — burny_tech · 2026-10-16
- Mathematicians publish explainer for the Egyptian fraction proof in OpenAI's proofs collection — burny_tech · 2026-10-16
- Robin Hanson: a neural network is almost 'a fractal of explanations' — burny_tech · 2026-10-16
- 'Neural networks really are super simple': a takes on features, concepts and connectionism — burny_tech · 2026-10-16