Apple research shows confidence-based discrete diffusion samplers hit theoretical limits
Apple ML Research · rss · 2026-10-02
Apple ML Research's paper "Limits of Confidence in Diffusion" analyzes discrete diffusion models, including remasking and uniform-state samplers, which write multiple token positions per step drawn from per-position distributions.
The authors prove that a step matches the training distribution only when the written positions are conditionally independent given the already-fixed tokens, and that no product of per-position distributions can match a dependent group of tokens. Since pixels, phonemes, and words exhibit inherent dependencies, confidence-based sampling strategies face fundamental limitations in these domains.
More from Research
- SAIL 2027 opens abstract submissions for AI in clinical care, deadline Jan 15 — zakkohane · 2026-11-19
- The academia LLM paradox: everyone writes with AI, nobody wants to read or be judged by it — wandedob · 2026-11-19
- Baseten hosts All-Gather, its first research conference on where AI's next gains will come from — baseten · 2026-11-19
- Interpretability solved—then what? Najoung Kim proposes filtering for recurrent constructs — najoungkim · 2026-11-19
- New preprint asks: if interpretability is "solved," what does it mean for the science of human language? — najoungkim · 2026-11-19
- Map of tabular foundation models shows only two are fully open — GaelVaroquaux · 2026-11-19