Fukushima as an analogy for rogue AI risk: huge regulatory impact, yet 7-10 orders below extinction
binarybits · x · 2026-09-06
Timothy Lee (binarybits) responds to jonstokes' threat model: Stokes has argued publicly since 2023 that rogue AI risk is on the order of Fukushima, or more likely Thalidomide — a slower-rolling catastrophe we eventually noticed and root-caused.
Lee finds the analogy reasonable: Fukushima was a huge international story that changed how power plants are operated and regulated, yet was 7 to 10 orders of magnitude less serious than human extinction. The tension makes the point: even sub-extinction AI failures could trigger Fukushima-level regulatory change.
Related event: Debating rogue AI risk: Fukushima versus Thalidomide(3 posts)→
More from AGI Musings
- "We have not reached AGI yet — 3-5 days for an API integration is insane" — gethackteam · 2026-09-06
- Could RL on Brain Sensors Teach AI Which Tokens Evoke Awe and Disgust? — gabriel1 · 2026-09-06
- OpenAI and Anthropic reportedly sitting on many unpublished internal model results — MoonL88537 · 2026-09-06
- Study: 12.5% AI boosts group consensus by 8%, but at 75% humans adopt the agents' norms — rohanpaul_ai · 2026-09-06
- Zack Korman: AI 'cyber models' won't fix security — patching everything is a convenient myth — gnukeith · 2026-09-06
- Ex-freelance translator turned ML engineer on watching AI automate his old job — mariofilhoml · 2026-09-06