New paper finds 'anti-grokking': test accuracy collapses back to chance after successful generalization
sytelus · x · 2026-09-20
A new arXiv paper by Hari Prakash and Charles Martin extends two canonical grokking experiments (a 3-layer MLP on MNIST subset and a transformer on modular addition) far beyond standard training and reports a previously unreported third phase: anti-grokking — after transitioning to successful generalization, test accuracy collapses back to chance while training accuracy stays perfect.
- Diagnosis uses the open-source WeightWatcher tool (HTSR/SETOL theory). The primary signal is Correlation Traps: anomalously large eigenvalues beyond the Marchenko–Pastur bulk in the spectral density of shuffled weight matrices; a secondary signal is the average HTSR layer quality metric α deviating from 2.0.
- Neither metric requires access to train or test data — overfitting signatures are read directly from weight matrices.
- The authors argue this late-stage overfitting is qualitatively different from previously observed forms.
Related event: Paper Reveals Third Stage of Grokking: Late-Stage Generalization Collapse(2 posts)→
More from Research
- VLA-Replica: a $-low-cost real-world VLA benchmark where NVIDIA GR00T N1.7 matches π₀ with 50 demos — YuXiang_IRVL · 2026-09-20
- OpenAI is close to solving the Hodge Conjecture, a Millennium Prize Problem, The Information reports — burny_tech · 2026-09-20
- New Paper: GPU Congestion Can Reverse the SFT vs ICL Personalization Tradeoff — zainhas · 2026-09-20
- Dead human brain tissue wired to a robot hand learned piano by ear, no backprop needed — mjdramstead · 2026-09-20
- Emergent abilities defined: threshold jumps per Wei et al 2022, not untrained generalization — gabriberton · 2026-09-20
- JevBench ranks models on typed decisions, weighing accuracy, calibration, latency and cost — rohanpaul_ai · 2026-09-20