NeurIPS 2026 Spotlight: encoder-readout decomposition explains grokking and double descent
s_y_chung · x · 2026-11-11
SueYeon Chung's team presents the NeurIPS 2026 Spotlight paper "Two Speeds of Learning", led by Chi-Ning Chou and Oscar Uzdelewicz with Neng-Chun Chiu and Yao-Yuan Yang, spanning Flatiron Institute, Harvard, and NYU.
- Question: why does test performance lag far behind train loss (grokking), or rise and fall while train loss keeps dropping (epoch-wise double descent)?
- Method: split the network into an encoder (representation learning) and readout (calibration) and track the two separately
- Finding: both phenomena arise from mismatched timescales between representation learning and readout calibration; using representational geometry, neural tangent kernels, and linear probing, the authors build a task-agnostic diagnostic framework across datasets and architectures
Related event: "Two Speeds of Learning" Paper Unifies Grokking and Double Descent(5 posts)→
More from Research
- Geodesic's NovaAtom-Lite hits 79% on FoldBench antibody-antigen prediction, beating AlphaFold 3's 48% — QuanquanGu · 2026-11-11
- Stanford AI100 releases 2026 report covering the scaling era and generative AI rise — TobyWalsh · 2026-11-11
- Prism capability extraction: specialists carved from Qwen 27B at <20% memory match full model — tokenbender · 2026-11-11
- Data Colada Flags Fraud in Fifth Francesca Gino Paper, No Forensics Needed — danielrock · 2026-11-11
- Fable 5.2 tops GDPx chip-design benchmark, but task authors see zero gains — ycombinator · 2026-11-11
- Fortnow asks Claude about complexity in the Aronson-Ambainis proof: Promise-BQP-complete — fortnow · 2026-11-11