Frank Nielsen generalizes Shannon entropy and KL divergence via pairs of quasi-arithmetic means
FrnkNlsn · x · 2026-10-27
Frank Nielsen shares his arXiv paper (2001.09660, published in Algorithms 2022) generalizing the α-divergence family using a pair of strictly comparable quasi-arithmetic means.
- Given two smooth strictly increasing functions f and g with f∘g⁻¹ strictly convex, one can define an (f,g)-entropy and a corresponding (f,g)-KL divergence (cross-entropy minus entropy).
- The limits α→1 and α→0 yield generalizations of KL and reverse KL divergences; the subfamily belongs to Csisár's f-divergences and can be rewritten as conformal Bregman divergences via monotone embeddings.
- Choosing f(x)=x and g(x)=log x recovers the classical Shannon entropy.
More from Research
- OpenAI claims Yang-Mills proof 4 days after Navier-Stokes: two Millennium Prize breakthroughs in one week — haider1 · 2026-10-27
- Gowers: OpenAI's math drop includes Kakeya proof; AGMAI urges release of Yang-Mills outputs — calabi_and_yau · 2026-10-27
- Protein LM embeddings + similarity-based label augmentation improve pKa prediction — bravo_abad · 2026-10-27
- New eLife study makes whole-brain mean-field models more biologically realistic — AnnaCiaunica · 2026-10-27
- GenXR-Bench: AI coding agents fail at XR apps, best model scores just 50.9% — rohanpaul_ai · 2026-10-27
- EmbodiedLab Benchmarks Robots That Must Act to Know in Simulated Labs — HexaFuture · 2026-10-27