rosinality Fixes Numerical Precision Issues in Softmax, Praised by Lucas Beyer
giffmana · x · 2026-10-23
Lucas Beyer (giffmana) boosted rosinality's work fixing a numerical precision problem in softmax. Softmax can overflow or produce NaN under extreme inputs, and this kind of low-level implementation fix has practical value for training and inference stability.
More from Research
- ICA rivals sparse autoencoders for LLM interpretability with 7–43× less memory — patrickmineault · 2026-10-23
- Paper proves swarm imitation is exactly single-agent RL, deriving Maynard-Cross Learning — mistlab · 2026-10-23
- MolmoBOT: 1.7M Simulated Trajectories Enable Real Robot Skills With Zero Real Data — chris_j_paxton · 2026-10-23
- Tohoku team launches Digital Hydrogen Platform with 30,000+ entries from 4,000 papers — BenBlaiszik · 2026-10-23
- Ropedia releases DONIC: one policy controls two dexterous robot hands, 86.79% success on unseen trajectories — liuziwei7 · 2026-10-23
- Tencent releases WeVisOne-VL: one VLM for detection, segmentation, OCR and matting — _akhaliq · 2026-10-23