Samuel Hammond: LLMs converge on brain-like structures, so take AI consciousness seriously
aran_nayebi · x · 2026-09-25
Samuel Hammond re-ups his case that frontier models like Claude and ChatGPT may have some form of subjective experience, arguing the claim is plausible enough to warrant research and precautionary steps (e.g. letting AIs quit distressing conversations).
- LLMs converge on brain-like structures during training
- Post-training appears to install a self-model
- Reward modeling may install valences for inner alignment
- If Attention Schema Theory holds, subjectivity is functional for long-horizon coherence and in-context learning; universality then gives reason to expect functional convergence toward consciousness if consciousness is functional
- He cautions that humans are agency-detection machines prone to projecting minds onto ambiguous stimuli, so the debate needs care
His conclusion: rather than declaring machine consciousness impossible, admit we don't know and study it.
More from AGI Musings
- Grady Booch: LLMs Only Resemble the Brain at Its Most Primitive Structures — Grady_Booch · 2026-09-25
- AI Agents Will Move Serious Money, But Nobody Has Answered the Liability Question — Master-Sprinkles-848 · 2026-09-25
- Krauss Podcast Chats With Sabine Hossenfelder on AI Breakthroughs, Nightmares, Dark Matter — skdh · 2026-09-25
- Materials PhD Uses AI to Map a Decade of Battery Literature, Questions Reading 100 Papers — Clean-Interview-5081 · 2026-09-25
- Bill Maher drops jokes for first time in 23 years to warn on AI; researcher says pause defeatism is untenable — wfithian · 2026-09-25
- Investor Stewart Alsop: AI agents are shattering an already fragile social trust — StewartalsopIII · 2026-09-25