Researcher Challenges Goodfire's Neuron Sparsity Paper

Rylan Schaeffer challenges Goodfire's claim that neurons are functionally sparse, arguing that ablating 92% of neurons for only a 2.9% loss increase is equivalent to discarding 50-90% of pretraining compute.

2026-11-14 ~ 2026-11-14 · 2 related posts

• Goodfire's 'modest' 2.9% loss increase equals discarding 50-90% of pretraining compute — RylanSchaeffer · 2026-11-14
• Researcher pushes back on Goodfire: 2.9% loss hike from ablation ≈ losing 50-90% of compute — RylanSchaeffer · 2026-11-14