Alignment researcher argues AI takeover is infeasible even with misaligned deployed models
JacquesThibs · x · 2026-09-25
In a discussion on misaligned deployed AI, the author agrees it's a concern and that a "train misaligned successor" path is plausible, but argues no one has laid out a detailed threat model for how it would happen. He expects takeover to be infeasible even if a deployed AI has misaligned goals, and says he found AI 2027 unconvincing on this point, likely intentionally papering over it.
More from AGI Musings
- Prediction: hospitals will keep their own tested open-model formularies in two years — MaziyarPanahi · 2026-11-12
- Packy revives Qwest CEO's 1998 'no bandwidth glut' quote for the AI oversupply debate — prestonpesek · 2026-11-12
- Stanford's James Zou joins The Economist podcast on AI agents in science — anshulkundaje · 2026-11-12
- a16z maps the B2A economy: AI agents are now most of web traffic and soon its buyers — venturetwins · 2026-11-12
- Jason: Anthropic is rebuilding every SaaS product into Claude—free for life — smtabatabaie · 2026-11-12
- Fleuret: if AI designs drugs as fast as it makes dumb videos, regulation will buckle — francoisfleuret · 2026-11-12