Score Centering fixes unstable RL for LLMs using an undergrad covariance identity
burny_tech · x · 2026-09-19
Researchers traced the root cause of RL instability when training and sampling policies differ — a common issue with LLM post-training. Instead of workarounds like matching numerics or importance sampling, they use the basic covariance identity E[AB] = E[A]E[B] + Cov(A,B) to cancel the destabilizing term directly. Their Score Centering method is simple to implement, compatible with existing approaches, and competitive in performance.
More from Research
- Anthropic says it may imminently hit RSP threshold for AI-accelerated R&D, releases provable inference work — tobyordoxford · 2026-11-04
- Google scanned 1,033 real household products, packed into a 345 MB GLB for three.js — victormustar · 2026-11-04
- These 10 GPU optimization papers are why your model runs fast today — NandoDF · 2026-11-04
- INT8 isn't the problem: I benchmarked samplers on INT8 flow models — YakovSava · 2026-11-04
- Karpathy Boosts Open Hacker News Dataset: 10M+ Posts Usable as a Personal Knowledge Base — huangyun_122 · 2026-11-04
- PixelCraft: A Multi-Agent Framework Targets Multimodal AI's Chart-Reading Failures — burkov · 2026-11-04