Researcher pushes back on Goodfire: 2.9% loss hike from ablation ≈ losing 50-90% of compute
RylanSchaeffer · x · 2026-11-14
Rylan Schaeffer challenges the opening claim of Goodfire's "Neurons as an Interpretable Basis," a companion to their viral interpretability paper.
The paper argues neurons are "already functionally sparse" because ablating the 92.2% weakest MoE neuron activations per forward pass causes only a "modest" 2.9% increase in loss. Schaeffer counters that 2.9% is brutal, not modest: under published pretraining scaling laws at modern compute budgets (10^21 to 10^26 FLOP), it's equivalent to discarding roughly 50-90% of pretraining compute.
The reason: loss falls with compute toward a floor, and compute can only buy down the reducible part above it. Using Marin's MoE scaling fit at frontier scale, only 0.34 of the 1.94 nats is reducible — so a 2.9% loss increase is far more damaging than it appears.
Related event: Researcher Challenges Goodfire's Neuron Sparsity Paper(2 posts)→
More from Research
- Measuring face reconstruction fidelity with AuraFace embedding cosine similarity — ostrisai · 2026-11-14
- 3DSceneBench: coding agents rebuild livable 3D rooms but top functionality score is just 42.1 — yizhu59 · 2026-11-14
- MIT team trains open 27B model with zero demos to score 18/30 on a real robot it never touched — songhan_mit · 2026-11-14
- New paper argues alignment is a system-level context property, not just model weights — rayanpal_ · 2026-11-14
- Claude invents the "Kairograph": turning its own attention mechanism into 125 photographic plates — Sunrise1927 · 2026-11-14
- Open-source live-cell hardware startup Aeon applies causal modeling to cell perturbation data — MaxUnfried · 2026-11-14