UIUC's Regularized Score Centering stabilizes async RL, tops all six agentic benchmarks
UIUC-CS · hf · 2026-11-18
A UIUC/CMU paper proposes Regularized Score Centering (RSC) to stabilize asynchronous reinforcement learning.
- Background: async RL boosts throughput, but stale-policy data and train/inference numerical mismatch destabilize learning
- SC's limits: score centering removes reward-baseline effects on updates and yields fewer extreme gradients than importance sampling (IS), but fails when the learner drifts far from the behavior policy
- Method: RSC adds a reverse KL penalty keeping the learner's next-token distribution close to the behavior policy, working with or without truncated importance sampling (TIS); theory shows KL regularization improves convergence guarantees
- Results: on dense and MoE models across math reasoning and agentic tasks (shopping, tool use, games, fully async software engineering), RSC+TIS ranks first on all six agentic benchmarks — +14.3% on AppWorld and 9.9x the strongest baseline on BabyAI
More from Research
- Voyager 1 Reaches One Light-Day From Earth, Signals Now Take 24 Hours — snowclipsed · 2026-11-18
- TCS Hiring Statement Urges Quality Over Quantity as AI Fuels Paper Arms Race — fortnow · 2026-11-18
- New paper: a sociocultural approach to linguistic creativity — abenitezburraco · 2026-11-18
- 27B Model Rewrites Its Own Weights Without Training or Gradients, Paper Claims — udmrzn · 2026-11-18
- From 'impossible' to a working Bluetooth beacon: open-source EDA chip design is real — IgorCarron · 2026-11-18
- AURA 2.0: a streaming video LLM that speaks first, calls tools, and grounds visually — Shuhan Li · 2026-11-18