Francis Bach Blog: Taming Exploding Variance of Exponential Means With Least Squares
BachFrancis · x · 2026-09-25
In a new blog post, machine learning researcher Francis Bach analyzes the exploding variance problem in estimating log-sum-exp functions via sampling: the relative error of estimating E[e^z] grows as (e^σ²-1)/n, exploding exponentially with σ, and taking logarithms doesn't help. He proposes a least-squares-based remedy and, as a bonus, derives a spectral closed-form formula for approximate multinomial logistic regression.
More from Research
- TorchUMM Library Accepted to NeurIPS 2026 — jindong_wang92 · 2026-09-25
- AgentArk, distilling multi-agent debate into a single LLM, accepted to NeurIPS 2026 — jindong_wang92 · 2026-09-25
- Deleting One Line Fooled Pangram: 100 Classic Texts Expose AI Detector's Truncation Weakness — GolinoHudson · 2026-09-25
- GRAIL Galleri trial of 142,000 fails to cut late-stage cancer incidence — EricTopol · 2026-09-25
- Microsoft's CASD: a coding agent reading full logs beats GEPA at prompt optimization by 5.7 points — dair_ai · 2026-09-25
- Microsoft's Taste-Bench: frontier agents pick the better direction only ~60% of the time — omarsar0 · 2026-09-25