Goodfire finds Neuron Communities: under 10 neurons can drive arithmetic, ALL CAPS in Llama-3.1-8B
niloofar_mire · x · 2026-11-18
Goodfire published new interpretability research introducing neuron community discovery, an unsupervised method for finding groups of neurons that repeatedly work together to implement common behaviors.
- Applied to Llama-3.1-8B, they found thousands of small, interpretable groups — often fewer than ten neurons — causally necessary and sufficient for tasks like grammatical agreement, arithmetic, ALL CAPS formatting, in-context copying, and date recall.
- Removing these groups substantially changes predictions, while keeping them preserves predictions even after ablating almost all other MLP neurons in the involved layers.
- Communities are discovered without specifying functions in advance. Work led by research fellow Igor Shilov with Imperial College London.
Related event: Goodfire Discovers Small Neuron Communities Driving LLM Behaviors(3 posts)→
More from Research
- UCLA hosts California AI Amplitudes school applying ML to quantum scattering amplitudes — KyleCranmer · 2026-11-18
- Looped layers don't help multimodal embeddings unless recurrence is pretrained (+4.23 points) — _reachsumit · 2026-11-18
- Ant Group's Repo-Premium shows BM25 matches dense retrieval in fair code-search tests — _reachsumit · 2026-11-18
- Kuaishou proposes GBPO, fixing GRPO's dead gradients and unbounded updates for generative recommenders — _reachsumit · 2026-11-18
- GainReason rewards each reasoning sentence by its predictive gain for generative recommendation — _reachsumit · 2026-11-18
- Training-free IN-Retriever uses embedding attention to navigate multi-hop RAG queries — _reachsumit · 2026-11-18