Berkeley findings: score centering alone removes drift in async RL, stacking corrections may not help
berkeley_ai · x · 2026-11-06
In a side project, Shu Liu and colleagues tested whether score centering (SC) is all you need for stable async RL, where practitioners often stack truncated importance sampling (TIS) or KL penalties on top. Results: SC alone removes drift and often matches on-policy performance; when it falls short, adding more corrections doesn't necessarily help. Ion Stoica amplified the findings.
Related event: Berkeley Experiments: Score Centering Alone Stabilizes Async RL(3 posts)→
More from Research
- Formal proofs of tens of millions of lines of code are now feasible, says Anthropic engineer — morqon · 2026-11-06
- Hugging Face releases FinePhrase slides: 486B-token synthetic pretraining dataset with 90 rewrite styles — NielsRogge · 2026-11-06
- Toby Ord: jump-plus-steeper-slope mixed hypothesis beats a steeper slope alone — tobyordoxford · 2026-11-06
- SynthFaces: Arc2Face diffusion plus flow matching build balanced synthetic face dataset — Sergey Karpukhin · 2026-11-06
- Stable AI proposes DPDM: frozen LLM representations plus shared predictor boost agent decisions — stable-ai · 2026-11-06
- DePue: the cited papers each describe very different phenomena — willdepue · 2026-11-06