Output Is Not Evidence: Why an LLM Saying It's Conscious Proves Nothing
ccerrato147 · x · 2026-09-27
A crisp epistemics rule for the AI consciousness debate: an agent's statements about itself are not evidence until independently verified. Citing Newman Nahas's critique of experiments that train Claude on documents about moral status and then interview it about its inner life, the author argues this is circular — training a model to emit tokens meaning "I am pregnant" doesn't make it pregnant.
To the objection "but it's truthful when it says it's an LLM": yes, because you verified that externally. Claims like "I feel sick" can't be checked from outside the screen, so they count for nothing. Whether a model is alive or conscious must be decided from other facts — or not decided at all. A test anyone can run today.
Related event: AI Claims of Consciousness Are Not Evidence, Argues Writer(2 posts)→
More from AGI Musings
- DHH: hand-writing code is no longer economically productive for most programmers — AccBalanced · 2026-09-27
- tszzl bets neural nets decompose into evolved symbolic systems before superintelligence — cephaloform · 2026-09-27
- No autobiographical memory, no consciousness: a new theory of the subject of experience — yeastsplainer · 2026-09-27
- tszzl bets neural nets will be shown to decompose into evolved symbolic systems before superintelligence — tszzl · 2026-09-27
- Dev bets top games in 20 years won't be made by AI — FanaHOVA · 2026-09-27
- Terry Tao on working with o1: like advising a mediocre but not incompetent grad student — burny_tech · 2026-09-27