How score centering is computed: topk(128) with a rho rescaling trick
nrehiew_ · x · 2026-10-16
nrehiew continues his breakdown of the Score Centering paper's implementation: to correct the bias, the cross-entropy gradient drift term must be subtracted from the gradient.
In practice the stop-gradient term is computed using only topk(128) of the vocab, with a coefficient ρ that rescales the top-k based on tail (non-top-k) probabilities so it still sums to 1 — making the correction cheap to compute.
Related event: RL training drift equals distillation toward the rollout sampler(2 posts)→
More from Research
- Is math just a language? A debate on why LLMs are good at mathematics — LucaAmb · 2026-10-16
- Alibaba open-sources OpenGCUA, a full-stack generalist computer-use agent; +19.6 points on OSWorld-v2 across 8 models — Tongyi-MAI · 2026-10-16
- CoDA co-evolves simulation and robot policies via shared failure memory, lifting OOD success from 5.3% to 27.6% — PLAN-Lab · 2026-10-16
- Cambridge NLIP seminar: Danae Sanchez Villegas on multimodal reasoning beyond final answers — nikaletras · 2026-10-16
- DKGI paper: agents write their own tools to search knowledge graphs, beating baselines by 6.7 Hit@1 — yuz9yuz · 2026-10-16
- ETH Zurich's LFD lets a frozen LLM teach drug-design generators natural-language goals — rohanpaul_ai · 2026-10-16