Extending SGD diffusion approximations to optimization over probability distributions
burkov · x · 2026-09-17
Modern ML increasingly optimizes over entire probability distributions, powering generative AI, Bayesian inference, and mean-field systems.
- In Euclidean settings, SGD is commonly approximated by Gaussian-noise-driven diffusion processes, making stochastic dynamics analytically tractable.
- Extending this principle to Wasserstein space has been hard due to its infinite-dimensional, nonlinear geometry.
- The article builds a rigorous mathematical foundation for replacing sample-driven randomness with tractable Gaussian fluctuations when optimizing over distributions.
More from Research
- SSAD 2026 talk maps the zoo of state representation designs for world models — abursuc · 2026-09-17
- Gowers responds to letter on maths and AI signed by 25 Fields medallists — tak3sh8 · 2026-09-17
- World models share one architecture — the tokenizer is where methods diverge — abursuc · 2026-09-17
- Anatomy of modern world models: a tokenizer compresses states, a module predicts the next — abursuc · 2026-09-17
- Speculative decoding: small draft model proposes tokens, big model verifies in one pass — HowDevelop · 2026-09-17
- Researcher proposes a journal for vibe-coded papers, with AI models as reviewers — peter_richtarik · 2026-09-17