New paper unifies Mirror Descent methods via trace-form entropies and deformed logarithms
FrnkNlsn · x · 2026-10-20
Frank Nielsen, Andrzej Cichocki, Toshihisa Tanaka and Sergio Cruces publish a journal paper (plus technical report) deriving a broad class of Mirror Descent (MD) and Generalized Exponentiated Gradient (GEG) algorithms from trace-form entropies defined via deformed logarithms.
- Improved convergence, robustness to vanishing/exploding gradients, and adaptability to non-Euclidean geometries via mirror maps
- Establishes deep connections to Amari's natural gradient, giving a unified geometric foundation for additive, multiplicative and natural gradient updates
- For the Tsallis, Kaniadakis, Sharma–Taneja–Mittal and Kaniadakis–Lissia–Scarfone entropy families, each induces a distinct Riemannian metric, yielding GEG algorithms that preserve the natural statistical geometry
- Tunable deformed-logarithm parameters enable adaptive geometric selection over classical Euclidean optimization
The framework unifies first-order MD methods under one information-geometric perspective based on generalized Bregman divergences.
More from Research
- Laude Institute launches FrontierCS 2: agents must beat deleted paper contributions blind — lateinteraction · 2026-10-23
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23