What if AI solves the hard problem of consciousness and we can't understand the answer?
ClankerCore · reddit · 2026-09-09
A discussion thread asks: what if AI cracks the hard problem of consciousness, but humans can't understand the answer? The post traces the landscape:
- Yudkowsky said yes a decade ago; Chalmers named the hard problem in 1995.
- Yoshua Bengio co-authored the 2023 "Consciousness in Artificial Intelligence" report; in 2025 he and Elmoznino published "Illusions of AI consciousness" in Science, warning that merely becoming convinced AI is conscious carries its own risks.
- Geoffrey Hinton goes further, arguing on 60 Minutes that neural networks genuinely understand and that multimodal systems already have what he calls "subjective experience" — a highly controversial position.
- In June 2026, Microsoft CSO Eric Horvitz and EPFL's Robert West published "A narrowing window to understand AI" in Science: human understanding of increasingly consequential AI is falling behind while AI models human psychology ever better — "we understand them less while they understand us more."
- This maps onto established debates on epistemic opacity and epistemic dependence: whether opaque systems can generate genuine scientific knowledge no human can fully reproduce.
More from AGI Musings
- Scooped Researchers Publish Less in Top Journals and Get 21% Fewer Citations — soumitrashukla9 · 2026-09-09
- Newton with 150 Hires? OpenAI Scooping Row Draws Leibniz Analogy — JMannhart · 2026-09-09
- Dev welcomes labs training on his code: better models benefit me too — intellectronica · 2026-09-09
- Tom Dietterich: AI safety hinges on novel, ill-defined tasks AI can't optimize — tdietterich · 2026-09-09
- Tom Dietterich clarifies: 'new knowledge' in AI debate means environment-interaction knowledge — tdietterich · 2026-09-09
- From GPT-4 failing basic addition to 10,000 agents solving a Millennium Prize problem in 3.5 years — MartinSignoux · 2026-09-09