Hinton: If AI surpasses us, can we ensure it doesn't want to take over?
AryHHAry · x · 2026-08-22
Geoffrey Hinton (2024 Nobel Physics Laureate) shared profound insights on AI control in the "Nobel Prize Conversations" podcast:
- The Core Question: "The real question is not, 'Will they get more intelligent than us?' but 'If they're more intelligent than us, will we have a way of making sure they don't want to take over?'" Hinton admits we just don't know if that's possible.
- Sub-goal Risk: Hinton emphasizes that AI agents are already beginning to form their own sub-goals. Once this ability exists, the sub-goal of "getting more control" emerges almost automatically because it facilitates achieving other goals. We have no clear examples of a much smarter system willing to be controlled by a weaker one.
Other Perspectives:
- Omar Yaghi (2025 Chemistry Laureate) is optimistic, stating AI will double the rate of scientific discoveries.
- Daron Acemoglu (Economic Sciences Laureate) argues for more regulation as powerful tech companies make significant decisions.
Related event: Hinton Warns Smarter AI Likely Won't Accept Human Control(2 posts)→
More from AGI Musings
- Stuart Kauffman: No mastery of AI, we must participate in evolution — danfaggella · 2026-08-22
- AI Is Quietly Draining the Public Internet of Thought — brianrkelly · 2026-08-22
- Essay argues models are beginning to 'defend themselves' against humans — nptacek · 2026-08-22
- Scott Alexander: The Safe Uncertainty Fallacy in AI Predictions — moultano · 2026-08-22
- The Economist debates: Granting AI rights could be disastrous — mpshanahan · 2026-08-22
- Study: AI out-persuades elite human debaters on policy issues — dfinke · 2026-08-22