"Two Speeds of Learning" Paper Unifies Grokking and Double Descent

A paper by Chi-Ning Chou and colleagues at Columbia University, "Two Speeds of Learning," was selected as a NeurIPS 2026 Spotlight. Led by Chi-Ning Chou and Oscar Uzdelewicz, with collaborators including Neng and others, it proposes a "two speeds" framework: throughout training, two kinds of learning happen in parallel—slow representation learning in the encoder and fast calibration in the final readout layer. The authors frame this as representation-readout decomposition, positioned as a top-down complement to circuit-level interpretability.

Confirmed

• The paper received a NeurIPS 2026 Spotlight; core authors are Chi-Ning Chou and Oscar Uzdelewicz, with SueYeon Chung on the team and posting a series of explainers on social media
• The framework uses two timescales to uniformly explain both grokking (delayed generalization) and per-epoch double descent
• The same measurement approach also identifies spurious generalization dynamics: previously reported MNIST grokking and label-noise double descent examples turn out to stem from representation degradation and readout misalignment under non-standard training recipes, and these effects vanish under standard training

Why it matters

• Grokking and double descent have long been viewed as mysterious or contradictory training phenomena; this work gives a unified mechanistic explanation, attributing them to the speed difference between representation learning and readout calibration
• Representation-readout decomposition offers interpretability research an analytical tool between macro behavior and micro circuits, and can distinguish genuine generalization from training-recipe artifacts—methodologically significant for understanding generalization itself

2026-11-11 ~ 2026-11-11 · 5 related posts

• [source] NeurIPS 2026 Spotlight: encoder-readout decomposition explains grokking and double descent — s_y_chung · 2026-11-11
• NeurIPS Spotlight: 'Two Speeds of Learning' unifies grokking and double descent — s_y_chung · 2026-11-11
• New paper: representation-readout decomposition unifies grokking and double descent, flags spurious generalization — s_y_chung · 2026-11-11
• [source] Grokking signatures in prior MNIST examples vanish under standard training, NeurIPS paper finds — s_y_chung · 2026-11-11
• [source] Representation-readout decomposition unifies grokking and double descent — s_y_chung · 2026-11-11