Neel Nanda's Classic Paper: Mechanistically Explaining Grokking in Three Phases
burny_tech · x · 2026-11-10
Reshared for Neel Nanda's birthday: his 2023 paper with Lawrence Chan, Tom Lieberum, Jess Smith and Jacob Steinhardt, Progress measures for grokking via mechanistic interpretability. The work fully reverse-engineers the algorithm learned by small transformers on modular addition—discrete Fourier transforms and trig identities convert addition into rotation on a circle—confirmed via activation/weight analysis and ablations in Fourier space. From this understanding the authors define progress measures splitting training into three continuous phases: memorization, circuit formation, and cleanup, showing grokking is not a sudden shift but gradual amplification of structured mechanisms followed by removal of memorizing components. The poster calls it a pure golden nerd-snipe that got them into neural geometry manifolds.
Related event: Neel Nanda's Classic Paper Decomposes Grokking into Three Stages(2 posts)→
More from Research
- Hazard-free Boolean circuits can be exponentially more expensive — and NP-hard to build — thomasahle · 2026-11-10
- Constructing hazard-free Boolean circuits proven NP-hard — thomasahle · 2026-11-10
- Unitree Go2 robot asks clarifying questions via Gemini and Bayesian active learning — DynamicWebPaige · 2026-11-10
- Visual Token Routing cuts computer-use agent inference cost up to 3.3x with 2.5-5.8x fewer tokens — wellecks · 2026-11-10
- SimFoundry auto-builds interactive 3D sim scenes from video for robot policy training — DynamicWebPaige · 2026-11-10
- CMU paper: one unmodified computer-use agent wins prizes across four embodied AI contests — wellecks · 2026-11-10