Polysemanticity explained: neurons 'grow' during training, absorbing weakly related concepts
basedjensen · x · 2026-10-17
dcolascione clarifies that @nickcammarata isn't claiming polysemanticity never happens—rather, with receipts, that its mechanism is neurons "growing" during training as they acquire ever-more-tentative matches to their main concept (e.g., a Titanic neuron slowly gaining weak activation for 19th-century steam power). Features are ill-defined like a coastline; semantic similarity predicting spatial co-location may explain why activations cluster.
More from Research
- LeanGraphSearch mines Mathlib dependency graphs, lifting proof success to 81% — kaiwei_chang · 2026-10-17
- Interpretability researcher suspects Claude and ChatGPT CoT show the same problematic behavior — BlancheMinerva · 2026-10-17
- Kornia ships open-source Rust SLAM pipeline, seeking battle testers — kornia_foss · 2026-10-17
- Perplexity decider v1.2 lands #3 on Decision Index as longer training erodes backbone skills — antoine_chaffin · 2026-10-17
- Perplexity traces decider v1.2 regressions to longer training, mitigates via model souping — antoine_chaffin · 2026-10-17
- Packing multiple questions into one forward pass saves compute but adds instability — antoine_chaffin · 2026-10-17