150 Async RL Experiments Show Score Centering Alone Often Suffices
Around 150 async RL experiments show that score centering alone is sufficient when the trainer stays close to the sampler, while adding truncated importance sampling or KL corrections may not help.
2026-11-06 ~ 2026-11-06 · 2 related posts
- Async RL study: score centering alone often matches on-policy, extra TIS/KL fixes don't reliably help — kastnerkyle · 2026-11-06
- ~150 async RL runs: score centering suffices as long as trainer stays close to sampler — brandondamos · 2026-11-06