Score Centering explained five ways: TIM drift, BPO, advantage baseline, STE, local score
青稞AI · wechat · 2026-10-18
A deep dive into Score Centering (SC) — proposed Sept 2026 by Martin Marek and Max Ryabinin, already merged into slime and verl — presenting five independent derivations:
- Original SC view: Training-inference mismatch (TIM) creates nonzero drift updates even with reward-free environments; the drift equals a distillation direction toward the sampler. SC removes it by centering scores and restores baseline-invariance.
- BPO view: SC's gradient matches the token-level surrogate of a critic-free objective derived from PMD's closed-form solution.
- Advantage view: SC emerges from wanting the sampler's exact value as baseline.
- STE view: SC is the straight-through estimator.
- Local distribution family view: SC is the true score of a constructed local family at its origin.
Implementation notes (verl): Full-vocab sampler distributions cost 600KB/position at 150k vocab in fp32, so top-k reconstruction is used with trainer-shaped tails; SC combines with TIS/MIS — exact IS already removes mean-score drift, but truncated weights reintroduce bias, and SC subtracts the residual mean.
More from Research
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23