Diffusion LM ablation: sampler choice moves 25 points, noise only 0.8
LucaAmb · x · 2026-10-27
Andrea Miele's team ran a controlled ablation on diffusion language models, which promise what autoregressive models can't: fixing their own mistakes. New methods usually change both the noise schedule and the sampler at once, making it unclear which matters. Testing on the hypersphere showed noise choice shifts results by just 0.8 points while the sampler accounts for 25 — the sampler dominates. They introduce Spherical Brownian Diffusion based on this finding.
More from Research
- Bret Victor's 2006 essay Magic Ink goes viral again: interactivity is a design crutch — round · 2026-10-27
- ICLR's new Informativeness review question wins praise: "What did I learn?" — CSProfKGD · 2026-10-27
- Citation analysis: neuroscience-AI bridge is real but narrow and tool-heavy — EricTopol · 2026-10-27
- HarborRL open-sources agentic RL: 200+ tasks, 40+ harnesses, one reproducible pipeline — Dear_Treat3688 · 2026-10-27
- Open-source Merlin model from JHU+Stanford auto-segments tumors on CT in one click — MaziyarPanahi · 2026-10-27
- Optimistic take on AI in math: researcher argues the old world is gone — Alex__007 · 2026-10-27