Double descent appears even in ordinary linear regression, tutorial paper shows
RylanSchaeffer · x · 2026-10-20
'Double Descent Demystified' began as a Stanford AI Alignment tutorial aiming to explain double descent with nothing beyond introductory linear algebra, isolating the three factors necessary to produce it. The attached figure shows three real datasets (California Housing, Diabetes, WHO Life Expectancy) plus one synthetic set all exhibit double descent in plain linear regression, with test loss spiking at the interpolation threshold. Meta-lesson: simplifying systems to their most basic pieces is tremendously valuable for identifying what really matters.
More from Research
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23