Aligning AI: hard engineering problem or existential threat? A three-hypothesis breakdown
sayashk · x · 2026-10-18
Philosopher Dan Williams proposes distinguishing three hypotheses when assessing whether misaligned AI poses existential risk: (1) alignment is easy, (2) we will build catastrophically misaligned AI capable of wiping us out, and (3) alignment is a hard but tractable iterative engineering problem. Costly misalignment incidents like Hugging Face support (2) over (1), but not clearly over (3). Hypothesis (3) better explains the full evidence—harmful misalignment remains rare despite millions using frontier models daily on complex agentic tasks—and plausibly carries a much higher prior.
More from AGI Musings
- Distillation may stunt Chinese labs' model personas, warns AI researcher — teortaxesTex · 2026-10-18
- Nick Bostrom on Roman Forum Podcast: "AI Is Already Conscious" — romanyam · 2026-10-18
- Prediction: Replacing a $50K Worker With AI Costs $100K and Delivers Negative ROI — sidjustice_ · 2026-10-18
- 'That is how early we are': most users still don't know agentic AI like Claude Cowork exists — dnlkwk · 2026-10-18
- Miles Brundage: No AI company is secure against state attackers — Miles_Brundage · 2026-10-18
- Fields Medalist Timothy Gowers: one-shotting a proof with ChatGPT means we don't understand it — haider1 · 2026-10-18