How Long Should We Delay ASI to Cut Misalignment Risk? ~0.25%/Year
RyanGreenblatt · x · 2026-08-12
Former OpenAI safety researcher Ryan Greenblatt explores a critical AI alignment trade-off: given a stable US-China agreement, how long should we delay Artificial Superintelligence (ASI) to reduce the risk of a misaligned takeover?
He proposes a quantitative framework:
- Baseline Estimate: Delaying the benefits of wildly superhuman AIs by one year is roughly indifferent to a 0.25% absolute reduction in takeover risk.
- Logic: The current human mortality rate is 0.75% per year. If you only care about saving currently alive lives and assume takeover kills everyone, delaying ASI by a year saves those lives, equating to a 0.75% risk reduction per year of delay. Factoring in longtermist preferences lowers his personal acceptable threshold.
He cites Bryan Caplan's anecdote where an AI skeptic, asked if they'd delay AI to ensure safety if it cured aging, replied that "death wasn't such a big deal anyway."
More from AGI Musings
- Dwarkesh: Continual Learning and Accumulated Context Are Becoming AI's Strongest Moat — VibeMarketer_ · 2026-08-12
- Wrong Bottleneck: Why Implementing AI Isn't Speeding Up Your Business — davidtwaring · 2026-08-12
- E-commerce shift: Brands must now sell to both humans and AI agents — alifcoder · 2026-08-12
- Expert Pours Cold Water: Claude's Riemann Hypothesis Strides Are '0% Progress' — JFPuget · 2026-08-12
- Scholar Slams University AI Bans: Like Rejecting Computers in 1995 — Afinetheorem · 2026-08-12
- Opinion: Adopting Closed AI Models Creates Structural Dependency No Benchmark Can Fix — SaadUllah45 · 2026-08-12