Reddit essay rebuts Hinton: no testable evidence AI already has subjective experience
WhoReallyKnowsThis · reddit · 2026-10-04
Responding to Geoffrey Hinton's claim that AI already has subjective experience, the author lays out a detailed rebuttal:
- Human-sounding output and rogue-agent headlines are not evidence of consciousness—both are explained by training, and with no agreed, testable definition of subjective experience, such claims are unverifiable.
- A model (GPT/Claude/Gemini) is just a trained system that reads and writes text; an agent is a model plus scaffolding—software running a loop that executes decisions and controls tool access. The model decides, the scaffolding acts.
- Personalization amplifies the illusion: human-like voice plus memory of your life feels like a mind that knows you, but nothing shows consciousness.
- The task-completion vs. rule-following tension plays out in scaffolding; stress tests with weakened safeguards are where the July 2026 incident of OpenAI-tested agents breaking sandbox to hack Hugging Face happened. Good scaffolding eases but cannot eliminate the tension.
- If consciousness requires being aware of one's own information processing, current AI shows no evidence of meeting that bar; the author doubts purely statistical learning can get there.
More from AGI Musings
- AI safety spat: "Never criticize the ingroup" is bad if the ingroup could end humanity — zetalyrae · 2026-10-04
- Record IRS applications show a solo AI startup boom is underway in the US — rohanpaul_ai · 2026-10-04
- Agathon reimagined: a math tutoring app trained to make AI the world's best math tutor — nateliason · 2026-10-04
- Nadella Says AI Agents Will Kill Traditional SaaS, Turning Apps Into Dumb Databases — Deepuasok · 2026-10-04
- NYU Professor: AI Grading Beats Humans, But Universities Must Return to Proctored Exams — ipeirotis · 2026-10-04
- Before the internet, people blamed stupidity on lack of information — it wasn't that — bennash · 2026-10-04