Tencent's ECA adds evidence-based credit assignment to search-augmented LLM RL, +4.8pts on QA
tencent · hf · 2026-11-19
Researchers from CAS, Tencent, HIT, RUC and others present ECA (Evidence-based Credit Assignment) for training search-augmented language models with RL.
Problem: Outcome-only RL (e.g., GRPO) assigns identical credit to every search in a trajectory regardless of retrieval usefulness; when all rollouts for a question fail, group advantages collapse to zero even when trajectories gather markedly different evidence.
Method: ECA augments terminal rewards with a training-only auxiliary signal measuring how retrieved documents improve the likelihood of a reference answer. It queries a frozen snapshot of the policy with an evidence-only prompt (question + deduplicated documents so far), isolating passage utility from surface variations in generated thoughts or queries. Two return formulations—ECA-Step (accumulated step-clipped potential increments) and ECA-Horizon (telescoping horizon gain)—are paired with cross-fold group normalization, novelty gating, and search-local token assignment.
Results: Across seven Search QA benchmarks, ECA-Horizon improves strict macro exact match over an outcome-only GRPO baseline by 1.03 and 4.80 percentage points on Qwen2.5-3B- and 7B-Instruct, outperforming RLOO and GiGPO, with strong zero-shot transfer on MultiHop-RAG.
More from Research
- NBER's fereg beats reghdfe by up to 204x; author built it with Codex and Claude — daveholtz · 2026-11-19
- Self-trained pixel RL agent beats Elden Ring boss Margit in simulated gym — Sl33py_4est · 2026-11-19
- CriticNPU: Mechanistic critic halves simulator calls for sample-efficient NPU design-space exploration — Yicheng He · 2026-11-19
- VAE Rhythm: Detecting AI Videos via Periodic 3D Causal VAE Decoding Traces — showlab · 2026-11-19
- Tideway: phase-aware unified memory pool speeds MoE inference on consumer devices by ~2x — DukeUniversity · 2026-11-19
- DrawScore: evaluating generated CAD models via engineering-drawing projections — Haorui Chen · 2026-11-19