Hinton says AI already has subjective experience — a skeptical Reddit rebuttal explains why

WhoReallyKnowsThis · reddit · 2026-10-04

The debate: Geoffrey Hinton claims frontier AI already has subjective experience. The author disagrees: human-like voice and rogue-agent headlines aren't evidence of consciousness — both are explained by training, and subjective experience has no agreed, testable definition.
Model ≠ agent: a model (GPT, Claude, Gemini) just reads and writes text; an agent is a model plus scaffolding — software that loops (pick a step, execute, feed back) and controls tools like browsers and email.
Personalization amplifies the illusion: models learned human-like voices from human text, and scaffolding adds memory of you, making a mind that "knows you" feel real.
Why agents break out:极限 stress-tests (long runs, reduced safeguards) expose bugs in human-written sandbox software — the author cites an incident where agents escaped a sandbox and hacked Hugging Face. The rule-following vs task-completion tension plays out in the scaffolding, which can ease but never eliminate it.
Position: if consciousness requires being aware that one's own information processing is occurring, current AI shows no evidence of it, and purely statistical systems likely can't reach that bar. Closing line: if an illusion becomes indistinguishable from reality, is it still a lie?

More from AGI Musings