Researcher argues fp32 or stochastic rounding for main weights is long-established practice
y0b1byte · x · 2026-11-18
In a reply to Google DeepMind researchers giffmana and mwulfmeier, y0b1byte argues that keeping main weights in fp32 or using stochastic rounding is a well-established convention across domains, responding to a discussion thread on numerical precision in training.
More from Research
- RADIA atlas tracks AI diffusion in Africa, where funding premium runs -9.4% vs +174% in NA — Afinetheorem · 2026-11-18
- AI biology startup Boltz raises $110M Series A, backed by Benchling's enthusiasm — GabriCorso · 2026-11-18
- Free online book: Error-Correction Coding and Decoding by Tomlinson et al. — burkov · 2026-11-18
- NOETIK partners with Memorial Sloan Kettering to build cancer biology world models — anshulkundaje · 2026-11-18
- bf16 breaking VLA fine-tuning sparks debate: fp32 or stochastic rounding already known? — giffmana · 2026-11-18
- Academic Research Skills for Claude Code hits 57.5k GitHub stars — adnan_hashmi · 2026-11-18