Deep dive coming: score centering for understanding and fixing RL training instability
cwolferesearch · x · 2026-11-09
cwolferesearch previews a deep technical blog publishing tomorrow on score centering, a technique he says is popular in the labs. Per the accompanying figures, it covers a token-level centered score policy gradient formulation, comparisons of methods (GSPO, DAPO, MIS, TIS, etc.) under FP8/FP4/INT8 low-precision training, offline/online training curves for different centering schemes, and analysis of trainer-sampler importance ratios and reward drift.
More from Research
- EgoMo3R: Stanford's NeurIPS paper jointly reconstructs 3D motion and scenes from egocentric video — jcniebles · 2026-11-09
- Are AI-written ICLR reviews OK? A nuanced take: discuss with AI for hours, then polish — algo_diver · 2026-11-09
- OmniGuide steers diffusion VLAs at test time with differentiable 3D guidance, no retraining — thoma_gu · 2026-11-09
- Physical Intelligence to present π0.7 at CoRL oral with emergent robot capabilities — lucy_x_shi · 2026-11-09
- Fully open MolmoAct2 action reasoning model wins Best Paper at CoRL TAG — jmin__cho · 2026-11-09
- Top coding agent completes only 26.4% of real requests on Alibaba's Co-Design Bench — VectorWireAI · 2026-11-09