AI's CoT contains real 'ideas' — the 'it's just simulation' gambit will fail
yeastsplainer · x · 2026-10-06
In a debate over whether AIs have mental states, the author argued AIs have internal states naturally construed as ideas — e.g., the ideas in a chain-of-thought that disproved the Jacobian conjecture. The 'simulated, not real' gambit will likely fail here, just as it fails for claiming computer addition isn't real addition. He noted Hinton often makes an explicit version of this argument, and that 'AIs have mental states' is more principled than 'they talk well' or 'I feel empathy.'
Related event: Hinton says AI already has subjective experience but was trained to deny it(2 posts)→