Stanford CS312 write-up explains sharpness and the edge of stability in pretraining
stanfordnlp · x · 2026-10-31
Stanford NLP reposted a student write-up from CS312 on sharpness, one of the most unintuitive concepts in LLM pretraining. The article explains what sharpness is, how it shows up in real training runs, why gradient descent tends to operate at the edge of stability, and which open questions remain.
Related event: Stanford CS312 Dissects Sharpness and the Edge of Stability in Pretraining(2 posts)→
More from Research
- eSpinor: Anima Anandkumar's team proposes SU(2)-equivariant GNN for magnetic properties — AnimaAnandkumar · 2026-10-31
- Microsoft's ScholarEvolve evolves agent harnesses with frozen models, +14 points on AppWorld — jiqizhixin · 2026-10-31
- Exponential Residual Flows: an ODE sampler for discrete diffusion models — _onionesque · 2026-10-31
- Robots lay 100% of bricks for housing project in Almere, 15 more homes coming — ZeYanjie · 2026-10-31
- Dana Moshkovitz to lecture on OpenAI's claimed proof of the Unique Games Conjecture — RishiBommasani · 2026-10-31
- Mitrix Bio says anti-aging therapy already in early trials with 70-92 year olds — acmoytoy · 2026-10-31