Neural nets aren't tangled knots: 90% of Qwen-32B neurons are monosemantic, says ex-OpenAI interp founder
burny_tech · x · 2026-10-16
Deedy amplifies Nick Cammarata's (founder of OpenAI's interpretability team, now at Goodfire) new research, calling it one of the most important AI results ever.
- Core finding: neural networks are not tangled knots — polysemanticity is wrong. You can study models' own neurons directly without "unfolding" them with a second model; 90% of Qwen-32B neurons have a single job.
- Why it matters: models now exceed humans at math, can break production software, and are reshaping white-collar work. This line of work may finally let us intentionally design LLM behavior and align it.
More from AGI Musings
- Physicist pushes back on ML community's acceptance that models may be beyond understanding — tszzl · 2026-10-16
- Ben Affleck calls AI 'just matrix multiplication'; Grady Booch echoes the skepticism — Grady_Booch · 2026-10-16
- Everyone has a precise AGI definition—until a model gets close to meeting it — VraserX · 2026-10-16
- Viral take: If AI gives us time back, its greatest gift is revealing what we mistook for the cost of living — r0ck3t23 · 2026-10-16
- "AI automation = slop, AI augmentation = super intelligence" — the line CEOs should see — jacob_posel · 2026-10-16
- Reddit take: Building a successor AI species would itself be humanity's purpose fulfilled — Flashy-Disk311 · 2026-10-16