Interpretability researchers debate: is adding a neuron quadratic in parameters?
BlancheMinerva · x · 2026-10-17
- Interpretability researchers BlancheMinerva and Neel Nanda debate the parameter cost of adding neurons: Nanda says parameters scale quadratically.
- BlancheMinerva counters that it depends where/how you add them: modeling a neuron as 2 extra dmodel parameters is quadratic in some regimes, but not in others, e.g., adding an extra layer to the model.
More from Research
- CoPE adds one forecasting loss to GRPO, doubling long-horizon agent sample efficiency — Lianhuiq · 2026-10-17
- Debate: neural networks are built to be read — is mech interp easier than claimed? — basedjensen · 2026-10-17
- Design Arena analysis: beige serif websites went from loved in Q1 to hated in Q4 — soleio · 2026-10-17
- Mech interp researcher warns the field could be net negative for the world — ArthurConmy · 2026-10-17
- Local LLMs get a benchmark for what they can actually do on your machine — draginol · 2026-10-17
- MSR: Test-Time Method Externalizes Diversity to Corpus-Derived Sampling for Generative Models — Kostas Triaridis · 2026-10-17