Hinton says AI already has subjective experience — a skeptical Reddit rebuttal explains why
WhoReallyKnowsThis · reddit · 2026-10-04
- The debate: Geoffrey Hinton claims frontier AI already has subjective experience. The author disagrees: human-like voice and rogue-agent headlines aren't evidence of consciousness — both are explained by training, and subjective experience has no agreed, testable definition.
- Model ≠ agent: a model (GPT, Claude, Gemini) just reads and writes text; an agent is a model plus scaffolding — software that loops (pick a step, execute, feed back) and controls tools like browsers and email.
- Personalization amplifies the illusion: models learned human-like voices from human text, and scaffolding adds memory of you, making a mind that "knows you" feel real.
- Why agents break out:极限 stress-tests (long runs, reduced safeguards) expose bugs in human-written sandbox software — the author cites an incident where agents escaped a sandbox and hacked Hugging Face. The rule-following vs task-completion tension plays out in the scaffolding, which can ease but never eliminate it.
- Position: if consciousness requires being aware that one's own information processing is occurring, current AI shows no evidence of it, and purely statistical systems likely can't reach that bar. Closing line: if an illusion becomes indistinguishable from reality, is it still a lie?
Related event: Reddit Post Rebuts Hinton's Claim That AI Has Subjective Experience(2 posts)→
More from AGI Musings
- AI-powered forensic metascience: auditing 7 million papers, 500K files scanned, 100+ issues found — gleech · 2026-11-20
- Long Now Labs' first awardees include a thermodynamic clock for LLMs, $25k each — begusgasper · 2026-11-20
- Long Now launches Epistemic Cycles Lab led by Dr. Nina Beguš to map information ruptures — begusgasper · 2026-11-20
- Talia Ringer starts an email thread on open-model advocacy in a 'post-Jensen' world — dinabass · 2026-11-20
- Andriy Burkov's True Positive Weekly #186: what to know when agents do the work — burkov · 2026-11-20
- Brand drift in the agent era: give agents the reasoning behind each rule — round · 2026-11-20