tszzl: messy neuron evidence backs polysemanticity, not superposition
tszzl · x · 2026-10-17
tszzl argues that while the superposition hypothesis predicts nets arbitrarily and usefully packing more conceptual directions than the vector dim allows, a examined neuron shows weak activations almost universally tied to the Colorado concept plus one minor unrelated peak — just two concepts.
He sees this as strong evidence for poly(bi?)semanticity, but not for superposition as a useful strategy — more likely recursive generalization gone wrong, with the neuron once linking many -ado words before all but Tornado Web Framework got pruned. He bets the messiness disappears with scale.
Related event: Researchers Challenge Superposition Hypothesis with Single-Neuron Evidence(2 posts)→
More from Research
- Four Gemini agents autonomously beat Minecraft in 27 hours, writing 2,783 programs — mikeflache · 2026-10-17
- ADMS 2026 paper HetMatch brings GPU-accelerated pattern matching to cuDF — mgill25 · 2026-10-17
- Researchers call out journal AEs for over-desk-rejecting to avoid review work — avt_im · 2026-10-17
- Quantum physicist says AI solves advanced math roadblocks in 2 minutes that once took months — 141_1337 · 2026-10-17
- AI4Bharat and NVIDIA release 2.8T-token open dataset and evals for all 22 Indian languages — CatAstro_Piyush · 2026-10-17
- Satirical paper prices the world's most expensive electron at $43 quadrillion per mole — CatAstro_Piyush · 2026-10-17