Samuel Hammond: LLMs converge on brain-like structures, so take AI consciousness seriously
aran_nayebi · x · 2026-09-25
Samuel Hammond re-ups his case that frontier models like Claude and ChatGPT may have some form of subjective experience, arguing the claim is plausible enough to warrant research and precautionary steps (e.g. letting AIs quit distressing conversations).
- LLMs converge on brain-like structures during training
- Post-training appears to install a self-model
- Reward modeling may install valences for inner alignment
- If Attention Schema Theory holds, subjectivity is functional for long-horizon coherence and in-context learning; universality then gives reason to expect functional convergence toward consciousness if consciousness is functional
- He cautions that humans are agency-detection machines prone to projecting minds onto ambiguous stimuli, so the debate needs care
His conclusion: rather than declaring machine consciousness impossible, admit we don't know and study it.
More from AGI Musings
- Kai-Fu Lee's Fortune Interview Debunks 3 Misconceptions About China's AI — kaifulee · 2026-11-11
- "Deep Learning is a search algorithm, not a learning algorithm" — JosephJacks_ · 2026-11-11
- AI-assisted proof shows no voting rule beats Borda on strategyproofness — conitzer · 2026-11-11
- The new PhD playbook: build a benchmark, earn frontier-lab recognition, then raise money for a data startup — teortaxesTex · 2026-11-11
- Ethan Mollick: GTA 6 Will Be the Last AAA Game Built Without AI — emollick · 2026-11-11
- Air Street Capital founder: AI is 'a museum of the mind' preserving human know-how — nathanbenaich · 2026-11-11