Same softmax numerical bug also affects KL regularizers in RL, researcher notes
heghbalz · x · 2026-10-28
Brandon Damos points out that the near-agreement numerical issue from the softmax/cross-entropy bug paper extends beyond distillation: it appears anywhere a KL regularizer is used, such as RL with maxπ E[r(y)] − β KL(π, πref). He adds that reverse KL and JSD have similar issues with analogous fixes, with a summary thread.
Related event: Google Paper Reveals Two Numerical Bugs in All Softmax Implementations(3 posts)→
More from Research
- Cohere Labs' CulturalRiddles: 6,200 Riddles in 51 Languages Expose AI's Cultural Blind Spots — Cohere_Labs · 2026-10-28
- Google's AI Mode feeds uncited Knowledge Graph biographies into answers, analysis shows — dejanseo · 2026-10-28
- DFR Framework Tackles Reward Fine-Tuning in Diffusion Models Without Losing Diversity — mmbronstein · 2026-10-28
- alphaXiv ships Mobile 2.0 with AI paper reader and one-click visual blogs — datawithsuman · 2026-10-28
- Researchers screened 175 Spanish books ahead of time and found multiple AI-generated positives — TuhinChakr · 2026-10-28
- Xiaohongshu and Fudan Propose DR-Rubric: Agent-Written Rubrics as RL Rewards — jiqizhixin · 2026-10-28