SPADER (EMNLP 2026): step-wise credit assignment and diversity-aware rewards lift multi-answer QA F1 across all benchmarks

机器之心 · wechat · 2026-10-18

SPADER, accepted to EMNLP 2026, addresses two misalignments in RL for multi-answer QA: trajectory-level advantage in GRPO gives coarse credit assignment, and F1 rewards under-incentivize hard-to-retrieve long-tail entities.

Its Step-wise Peer Advantage aligns parallel trajectories by decision step and uses group future returns as a baseline (no critic or external evaluator), while a diversity-aware exploration reward applies novelty dilution plus tool costs so termination behavior emerges from the reward structure. SPADER ranks first in F1 on four benchmarks across Llama3.1-8B and Qwen3-8B, beating PPO/GRPO by up to 86.1% on QAMPARI without extra search overhead.

Original post →

SPADER (EMNLP 2026): step-wise credit assignment and diversity-aware rewards lift multi-answer QA F1 across all benchmarks

More from coding & agent