Stop-gradient on sampling yields a principled but high-variance update direction
mblondel_ml · x · 2026-11-17
Part three of Blondel et al.'s paper thread: applying a stop gradient to the sampling procedure yields a theoretically principled "moment matching" update direction, but it suffers from high variance in practice. See the thread's first post for the full analysis.
More from Research
- Zuse: decision models for 27 European languages hit 87.5% accuracy at 27B — bineric · 2026-11-17
- Copenhagen postdoc chance at biomaterials × machine intelligence, deadline Dec 6 — _onionesque · 2026-11-17
- $1.1M Templeton grant funds Global Virtues Atlas of moral beliefs for AI alignment — MohammadAtari90 · 2026-11-17
- Dev proposes swarm of specialist diffusion models for code where invalid syntax is unrepresentable — SubtleGradient · 2026-11-17
- Economists warn peer review is collapsing under 99% AI-generated papers and referee reports — Afinetheorem · 2026-11-17
- Meta's Self-Improving Memory: agents discover their own memory skills and distill them into weights — jaseweston · 2026-11-17