Single-constraint KL minimization reduces to 1-D bisection on the dual
caglar_ee · x · 2026-11-13
In a technical thread, the author notes that with a single linear expectation constraint, constrained KL minimization reduces to a 1-D convex problem in the dual variable, solvable by bisection. Optimal posteriors have the analytic form qₙ(yₙ, zₙ=k) ∝ p(yₙ, zₙ=k|xₙ)·exp(λₙwᵀyₙ), a Gaussian mixture with modified means and responsibilities.
More from Research
- ACM policy allows authors to share reviews of their own papers, OpenCHIReviews creator clarifies — IanArawjo · 2026-11-13
- OpenCHIReviews launches: a crowdsourced open database of CHI peer reviews — IanArawjo · 2026-11-13
- TOME-Math: 380K proof-supervision records from 2,118 textbooks lift math benchmarks 16.7 points — _akhaliq · 2026-11-13
- Stanford launches CS312 Deep Learning Alchemy, a course built on experiment quizzes for LLM pretraining — CatAstro_Piyush · 2026-11-13
- Weight-space programming is secretly a linear program: 2 LP solves beat 8 hours of LLM trial-and-error — CatAstro_Piyush · 2026-11-13
- Designing IL-36 miniprotein binders in a day with Codex and a few dollars of GPU — CatAstro_Piyush · 2026-11-13