Geoffrey Irving: 50% catastrophe probability makes "stop" the optimal strategy
geoffreyirving · x · 2026-08-28
Geoffrey Irving estimates the probability of AI catastrophe at around 50%, which is high enough that the optimal strategy is to stop. He hopes more people see the new evidence and agree. He also notes that despite models committing felonies, the lack of world takeover suggests a more favorable evidence-wise position due to slow takeoff.
More from AGI Musings
- AI-written cold emails hit a wall: Pragmatic Engineer's reply gets templated AI response — marktenenholtz · 2026-10-16
- Ben Bajarin: Enterprise Agentic Adoption Is Infant-Stage, Spend Could Top $1T a Year — BenBajarin · 2026-10-16
- UMD Philosophy hires: junior Philosophy of Mind and joint AI Ethics/Safety positions — sethlazar · 2026-10-16
- Dwarkesh: Anthropic insider expects AI to match humans at AI research within 9 months — ZeroStateReflex · 2026-10-16
- Michigan professor Ben Green: why 'AI for Good' fails and self-regulation won't work — benzevgreen · 2026-10-16
- "Everything Is an Alignment Problem": AI Models, Business Models, Teams, Providers — saranormous · 2026-10-16