Hinton: Superintelligence Alignment Should Be Trained In, Not Locked Down

victor_explore · x · 2026-09-22

Geoffrey Hinton argues almost nothing more intelligent is ever controlled by something less intelligent — the one exception being a baby controlling its mother by crying. He says superintelligence must be built the same way.
Commentator victorexplore reframes this: alignment shouldn't be built as a lock (a permissions problem) but as a preference you train into the model — making it fundamentally an eval problem.
The thread pushes back on mainstream guardrail-and-permission approaches, arguing for training preferences rather than restricting capabilities.

More from AGI Musings