Training recipe shapes neuron interpretability: GPT-2's dropout may skew findings
tszzl · x · 2026-10-26
- A PSA for neuron interpretability researchers: the model's training recipe matters a lot, so don't over-generalize findings.
- GPT-2 appears to have been trained with dropout (modern LLMs are not), which seems to increase redundancy and possibly polysemanticity while reducing superposition.
- Other choices — SwiGLU vs GELU MLPs, optimizer, weight decay — likely matter too.
- Much existing interp work uses older/toy models that may not be representative. The author is starting a project to characterize how these choices affect neuron interpretability and invites collaborators.
More from Research
- Glorius Group Releases PRISM, a Bidirectional NMR Foundation Model — CatAstro_Piyush · 2026-10-26
- Surprising result: character LoRA trained on the SFT model gives the best likeness — 1or4s · 2026-10-26
- DeepSeek 1017 checkpoint lands on FrontierCS 2 with a small score jump — scaling01 · 2026-10-26
- Gandalf paper: KV cache offloading wears out SSDs in months; attention-aware admission extends life 7.3x — ThePrimeClock · 2026-10-26
- Baidu's LeWAM paper added its name 17 days after submission, right after rival LeWAM papers appeared — randall_balestr · 2026-10-26
- RAVEL: training 210 MLLMs to build a strong baseline for picking vision encoders — jiqizhixin · 2026-10-26