Loss Functions Are Scientific Assumptions: MSE Implies Gaussian Noise, Cross-Entropy Implies Bernoulli
bravo_abad · x · 2026-09-11
The author argues that choosing a loss function in ML is itself a scientific assumption about how data were generated: minimizing MSE is equivalent to maximum likelihood under additive Gaussian noise, while cross-entropy emerges naturally from a Bernoulli model. Changing the loss changes which errors the model treats as costly and the implied statistical picture of the experiment. For scientific ML, the key question isn't just which model to use, but what noise and error assumptions you're making — the loss function is part of the scientific model. The idea comes from his 2026 book on AI for science students.
More from Research
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11
- Skyfall GS Uses Flux to Refine Gaussian Splatting, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11
- Could 10k agents discover learning methods beyond backprop, or just tweak existing ones? — SeunghyunSEO7 · 2026-09-11