Neel Nanda's grokking paper: reverse-engineering emergent behavior into three phases
burny_tech · x · 2026-11-10
A share of the paper "Progress measures for grokking via mechanistic interpretability" (arXiv:2301.05217) by Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith and Jacob Steinhardt.
- Fully reverse-engineers the algorithm learned by small transformers on modular addition: discrete Fourier transforms and trig identities converting addition to rotation on a circle, validated via activation/weight analysis and Fourier-space ablations.
- Defines continuous progress measures that split training into three phases: memorization, circuit formation, and cleanup.
- Conclusion: grokking is not a sudden shift but the gradual amplification of structured mechanisms followed by removal of memorizing components — a blueprint for studying emergent behavior via mechanistic interpretability.
Related event: Neel Nanda's Classic Paper Decomposes Grokking into Three Stages(2 posts)→
More from Research
- Hazard-free Boolean circuits can be exponentially more expensive — and NP-hard to build — thomasahle · 2026-11-10
- Constructing hazard-free Boolean circuits proven NP-hard — thomasahle · 2026-11-10
- Unitree Go2 robot asks clarifying questions via Gemini and Bayesian active learning — DynamicWebPaige · 2026-11-10
- Visual Token Routing cuts computer-use agent inference cost up to 3.3x with 2.5-5.8x fewer tokens — wellecks · 2026-11-10
- SimFoundry auto-builds interactive 3D sim scenes from video for robot policy training — DynamicWebPaige · 2026-11-10
- CMU paper: one unmodified computer-use agent wins prizes across four embodied AI contests — wellecks · 2026-11-10