AI can simulate how a self talks, but not what a self is
gerardsans · x · 2026-10-21
Replying to a discussion on higher-order theories of consciousness, the author argues that training data contains only first-person report samples with no information about internal states, making sampling superficial and structureless. Example: training an AI on someone's email traffic recovers only what appears in the emails—never their life experiences or thought processes. An AI can simulate how a self would reply in chat, but cannot simulate what a self is; understanding these technical limits prevents assuming impossible capabilities.
More from AGI Musings
- Polymarket puts just 7% odds on OpenAI pausing AI training this year — Polymarket · 2026-10-21
- Daniel Lemire: AI-generated papers are exposing academia's 'Profzi scheme' — lemire · 2026-10-21
- Universities block thesis downloads while each dissertation averages barely a handful of lifetime reads — generativist · 2026-10-21
- Berkeley wins Schmidt Sciences HAVI grant to decode whale and parrot communication with AI — begusgasper · 2026-10-21
- 3Blue1Brown says he'll keep making videos long after LLMs can generate clear explainers — 3blue1brown · 2026-10-21
- Talos researcher: AI agent attacks today are loud pentests — quiet swarms are the real threat — matthew_d_green · 2026-10-21