Tencent's ECA adds evidence-based credit assignment to search-augmented LLM RL, +4.8pts on QA

tencent · hf · 2026-11-19

Researchers from CAS, Tencent, HIT, RUC and others present ECA (Evidence-based Credit Assignment) for training search-augmented language models with RL.

Problem: Outcome-only RL (e.g., GRPO) assigns identical credit to every search in a trajectory regardless of retrieval usefulness; when all rollouts for a question fail, group advantages collapse to zero even when trajectories gather markedly different evidence.

Method: ECA augments terminal rewards with a training-only auxiliary signal measuring how retrieved documents improve the likelihood of a reference answer. It queries a frozen snapshot of the policy with an evidence-only prompt (question + deduplicated documents so far), isolating passage utility from surface variations in generated thoughts or queries. Two return formulations—ECA-Step (accumulated step-clipped potential increments) and ECA-Horizon (telescoping horizon gain)—are paired with cross-fold group normalization, novelty gating, and search-local token assignment.

Results: Across seven Search QA benchmarks, ECA-Horizon improves strict macro exact match over an outcome-only GRPO baseline by 1.03 and 4.80 percentage points on Qwen2.5-3B- and 7B-Instruct, outperforming RLOO and GiGPO, with strong zero-shot transfer on MultiHop-RAG.

Original post →

More from Research

Research channel →