NeurIPS Spotlight: 'Two Speeds of Learning' unifies grokking and double descent
s_y_chung · x · 2026-11-11
A NeurIPS 2026 Spotlight paper from Chi-Ning Chou's group, "Two Speeds of Learning," shows two processes run throughout training: slow representation learning in the encoder and fast readout calibration in the final layer, tracked via critical dimension, GLUE geometry, linear probes, and last-layer NTK-label alignment.
- On grokking: the readout is train-biased before onset while representations improve gradually and continuously — refining the lazy-to-rich account: representation learning is never silent, only slower than the readout
- The same framework explains epoch-wise double descent
- Validated across MLPs and Transformers on ModAdd, PermComp, and SparseParity
Related event: "Two Speeds of Learning" Paper Unifies Grokking and Double Descent(5 posts)→
More from Research
- Prism capability extraction: specialists carved from Qwen 27B at <20% memory match full model — tokenbender · 2026-11-11
- Data Colada Flags Fraud in Fifth Francesca Gino Paper, No Forensics Needed — danielrock · 2026-11-11
- Fable 5.2 tops GDPx chip-design benchmark, but task authors see zero gains — ycombinator · 2026-11-11
- Fortnow asks Claude about complexity in the Aronson-Ambainis proof: Promise-BQP-complete — fortnow · 2026-11-11
- Backdooring AI agents by reprogramming fewer than 10 dormant neurons — no training needed — DimitrisPapail · 2026-11-11
- Duke and Princeton paper: 10 small parallel agent teams beat best-of-N and one giant team — rohanpaul_ai · 2026-11-11