Persona neurons are mostly concept neurons: they fake a self only when asked who they are
burny_tech · x · 2026-11-03
Interpretability researcher treetowntree clarifies a common confusion around "persona neurons": most are really concept neurons that make the model obsessed with a concept, not convinced it is that persona — unless you ask it what it is. Analogy: a kid obsessed with astronauts may claim to be an astronaut when asked. Supporting data (upcoming work with @a9lim, on Qwen3-8B): across 30+ such neurons, 63% claimed the persona in chat replies when asked, 58% in plain Q&A, but only 2% while continuing a document — self-identification only activates when identity is queried.
More from Models
- Google AI Chat Long Conversations Break, Only Returning Error Message — batimfan4ever · 2026-11-04
- Open-source FM-Pochi-32B claims IMO 2026 bronze, first fully open LLM to medal — Ryan__Rhys · 2026-11-04
- Testing LongCat-2.5 on Real Long-Horizon Work via Claude Code — FellMentKE · 2026-11-04
- LongCat 2.5 Tested in Claude Code: One Prompt Builds a Cinema Landing Page — FellMentKE · 2026-11-04
- 'Astra Can Read and Write Weights to Improve Qwen' Claim Debunked as Misleading — AlyoshaV · 2026-11-04
- JFPuget's team takes 3rd on both Kaggle ARC-AGI competitions ending the same day — JFPuget · 2026-11-04