Lab insiders know alignment is unsolvable but can't say it until the Overton window shifts
danfaggella · x · 2026-10-30
Dan Faggella argues that people inside AI labs and big tech only voice obvious truths once the Overton window allows it.
His claims:
- The field now mostly admits AGI risk is real and deserves serious attention.
- But many publicly repeat things they privately know are false: that alignment can be "solved," and that humans will live happily alongside robots as eternal "partners" — implausible under exponential AGI takeoff.
- Speaking honestly too early looks "bad and scary" and carries career risk: OpenAI's roon only openly discussed AGI slowdown after the Pacing post, despite having seen the risks long before.
- A DeepMind employee openly saying alignment is impossible in principle would make the lab look like "bad guys building bad AGI," and competitors would exploit that perception — so everyone maintains the lie until a critical threshold.
He predicts lab people will eventually be able to say "alignment won't be solved" and "humans must coevolve or probably perish" — but until then, the lies will keep degrading the discourse.
More from AGI Musings
- Garrett predicts customer support collapse as bots flood and firms ignore requests — GabGarrett · 2026-10-31
- Jacques Thibodeau: treating AI 'solutions' as solved pushes us toward failure worlds — JacquesThibs · 2026-10-31
- YC's Garry Tan: AI turns code into a rushing river—much of software must be rewritten — garrytan · 2026-10-31
- 'RAG ability is RLable' — the insight reshaping how models learn retrieval — inductionheads · 2026-10-31
- ControlAI's Connor Leahy: 'If we continue on this path, the default outcome is we don't make it' — notkilleveryoneist · 2026-10-31
- Prediction: Claude + Instagram will mint more corporate millionaires in 2026 than crypto did in 2014 — ahuja_priyank · 2026-10-31