Goodfire's 'modest' 2.9% loss increase equals discarding 50-90% of pretraining compute
RylanSchaeffer · x · 2026-11-14
GoodfireAI's new paper 'Neurons as an Interpretable Basis' claims neurons are 'already functionally sparse', citing that ablating the weakest 92.2% of MoE neuron activations raises LM loss by only 2.9%. Rylan Schaeffer pushes back: a 2.9% loss increase is brutal. Under published pretraining scaling laws at 10^21–10^26 FLOP, it equates to discarding roughly 50–90% of pretraining compute (59% at 10^22 FLOP, 80% at 10^26). The reason: pretraining loss falls toward a floor and compute only buys down the part above it — in Marin's MoE fit at frontier scale, only 0.34 of 1.94 nats is reducible, so 2.9% of total loss is 17% of the reducible loss. Small loss deltas can massively misrepresent the true cost of sparsity ablations.
Related event: Researcher Challenges Goodfire's Neuron Sparsity Paper(2 posts)→
More from AGI Musings
- Recursive self-improvement has no single starting moment — Cosmos Institute essay dissects the intelligence explosion — sebkrier · 2026-11-14
- Gemini helps WSJ reporter crack her 20-year medical mystery — vivnat · 2026-11-14
- 77% of clinical drugs target the same 20% of popular targets—AI could fix this — jacobkimmel · 2026-11-14
- Boomi AI chief on agentic truth: enterprises need a context layer, not just clean data — jonerp · 2026-11-14
- Revisiting 'Cybernetic Serendipity': the vocabulary of digital art's origins — Merzmensch · 2026-11-14
- 'Living in a different world': an AI researcher's dispatch from the psychosis frontier — illscience · 2026-11-14