Meta's Self-Credit: Dense Rewards from Sparse RLVR via Intra-group Rescaling
meta · hf · 2026-10-21
Meta releases Self-Credit, an end-to-end agentic RL method tackling credit assignment under sparse RLVR rewards in long-horizon tasks by constructing dense supervision from outcome rewards.
Built on group-based RL with three components:
- Self-generated credit: the policy emits summaries citing earlier turns containing critical evidence during rollouts.
- Intra-group rescaling: cited actions are compared across successful and failed rollouts of the same prompt, using group agreement/disagreement to remove undeserved credit.
- Advantage reweighting: rescaled credit weights update advantages of cited turns.
Results: on BrowseComp-Plus, +37.0 and +36.2 points over base Qwen3 8B/32B, beating the strongest agentic RL baselines by 8.5/5.6 points; +4.9 on GAIA and +4.7 on xBench-DeepSearch. Ablations show rescaling on failed rollouts matters across scales.
More from coding & agent
- 90% of big-company knowledge lives in heads, and "company brain" startups are in for pain — gabriel1 · 2026-10-21
- EAU: Co-evolving user simulator, environment, and RL setup for training LLM agents — UAlberta · 2026-10-21
- Notion details its Auto model router: Elo-based evals, 62-93% cheaper than flagship models — ivanhzhao · 2026-10-21
- Dev sets up automation so agents proactively pick up stale issues and PRs — vikvang1 · 2026-10-21
- Tamper-proof agent logs still can't prove the agent did the job: who reads the record back? — ZealousidealAct9661 · 2026-10-21
- One prompt: Claude builds an animated mech spider in Blender via MCP, first try — sidahuj · 2026-10-21