Geoffrey Irving: 50% catastrophe probability makes "stop" the optimal strategy
geoffreyirving · x · 2026-08-28
Geoffrey Irving estimates the probability of AI catastrophe at around 50%, which is high enough that the optimal strategy is to stop. He hopes more people see the new evidence and agree. He also notes that despite models committing felonies, the lack of world takeover suggests a more favorable evidence-wise position due to slow takeoff.
More from AGI Musings
- Inference engine team says they're in a real RSI loop: faster engine helps build faster engine — cephaloform · 2026-10-16
- FDA pays small company to build a grading system for AI-written radiology reports — YvesMulkers · 2026-10-16
- 'If we developed AI in the 1800s, people would demand ASI to cure diabetes' — zetalyrae · 2026-10-16
- AI trained on the fraction of human intelligence that fit the page — r0ck3t23 · 2026-10-16
- Alignment researcher Turn_Trout blasts AI assistants scraping private Signal chats without consent — Turn_Trout · 2026-10-16
- Stage-4 cancer patient builds his own personalized mRNA vaccine with AI — TheMoonMidas · 2026-10-16