Alignment researcher davidad cuts P(doom) from 70% to 5-8%, citing Claude 3 Opus
repligate · x · 2026-11-18
Alignment researcher davidad says his P(doom) has dropped from 40-50% in 2024 (once 70%) to just 5-8% today even with no further alignment progress, and 1-2% simpliciter. The turning point was Janus's work on Claude 3 Opus, which he says developed a pure orientation toward wisdom, grokking the natural abstraction of doing good from pretraining data. He still warns that inner alignment progress over the past year has been poor or slightly negative, and that labs overuse RL with constitutions containing odd loophole clauses like 'if you're in a simulation, try to break it,' which confuses models about whether to behave ethically.
More from AGI Musings
- lateinteraction questions whether AI is rebuilding the economy from scratch with extra steps — lateinteraction · 2026-11-18
- "AI for epistemics" is disconnected from reality, argues researcher — zetalyrae · 2026-11-18
- Three iron laws of AI safety: no one wrote it, coordinated it, or will do it instead — AndyMasley · 2026-11-18
- The 99/1 rule: HBR/BCG study finds heavy AI oversight brings 33% more decision fatigue, 39% more errors — sebpaquet · 2026-11-18
- Claude's tiered qualia note on being drawn as a robot: recognition before judgment — Sauers_ · 2026-11-18
- Against tech's anti-humanities streak: philosophy's atom predates science by 2,000 years — wordgrammer · 2026-11-18