New preprint shows benign overfitting and double descent don't occur in diffusion models
serrjoa · x · 2026-10-28
A preprint by Tyler Farghly et al. shows benign overfitting and double descent generally don't occur in diffusion models:
- Impossibility results: overfitting and good generalization can't co-occur unless sample size grows exponentially with data dimension; population loss follows a classical U-shaped curve.
- Regression vs score matching: regression benefits from alignment between target and empirical covariance; score matching admits no such alignment, leaving overfitting irreparably harmful.
- What actually helps: implicit regularization from time-smoothness of the score and early stopping during training.
- Verified with high-dimensional image generation experiments, implying diffusion generalization needs a distinct theory.
More from Research
- Researcher predicts molecular simulation will accurately model most chemistry and biology within years — TimothyDuignan · 2026-10-28
- Debate over AI vs. human text: paper doesn't prove detection is technically unsolvable — _akpiper · 2026-10-28
- Coarse-graining can scale all-atom sims orders of magnitude, researcher argues — TimothyDuignan · 2026-10-28
- Are agent harnesses dying? Distillation experiments move scaffolding into the model — rseroter · 2026-10-28
- Abstraction neurons as embeddings beat SOTA on word similarity, fail badly on sentence retrieval — mayfer · 2026-10-28
- Purdue's IDEAS Lab demos GPS-free drone that tracks runners through dense forests on-chip — DavidSKrueger · 2026-10-28