ICML paper: popular information-theoretic measures poorly estimate uncertainty
gleech · x · 2026-08-19
A thread asking why Shannon's entropy should measure "uncertainty" sparked discussion, with BrantonDeMoss recommending Bickford Smith et al.'s ICML 2025 paper "Rethinking Aleatoric and Epistemic Uncertainty".
The paper argues the aleatoric-vs-epistemic framing is incoherent and insufficiently expressive for the quantities researchers care about. It offers a decision-theoretic perspective defining uncertainty as expected loss under the Bayes-optimal policy, rigorously linking uncertainty, predictive performance, and statistical dispersion. It further shows popular information-theoretic quantities can be poor estimators of what they purportedly measure, while explaining how they remain useful for guiding data acquisition.
More from Research
- Open-source Ornith-1.5 hits SOTA with self-improving strategies — aftahi_ai · 2026-08-19
- Criticizing Wildchat data distribution as representative of real-world use — yoavartzi · 2026-08-19
- Study finds data scaling in scientific AI is not always smooth — bravo_abad · 2026-08-19
- FAR system aims to exponentiate mathematical discovery via dynamic compute allocation — wellecks · 2026-08-19
- Matryoshka LM Suites: Train model suites in a single run — yoavartzi · 2026-08-19
- Moderna's AI-designed personalized mRNA cancer therapy succeeds in Phase 3 trial — kimmonismus · 2026-08-19