A fusion-reactor satire of frontier AI: 30% extinction risk, 'just give us 20 trillion evaluations'
Conscious-Neat-5015 · reddit · 2026-09-19
A Reddit thought experiment, half-satirical: imagine a startup building a new power reactor producing 1 petawatt (global output is 20 terawatts), with a 30% chance of an explosion that ends humanity.
The author uses the scenario to parody frontier AI race rhetoric:
- "We can't control it yet, but the upside reverses climate change and colonizes Mars"
- "Going slow matters, but competitors will rival us, any design could trigger it"
- "We keep our secret so rivals can't win"
- "We're the good guys — approving us only requires 20 trillion evaluations"
The post is effectively a parody of the 'dangerous technology + race logic + we're the responsible ones' framing of AI safety debates.
More from AGI Musings
- eigenrobot shrugs at AI 'independent proof' plagiarism scenario raising ethics debate — eigenrobot · 2026-09-19
- APA warns: kids and teens are turning to AI for friendship, advice, and emotional support — AnnaCiaunica · 2026-09-19
- Falsifiable test proposed for what triggers model pain directions — EigenGender · 2026-09-19
- "I'm a single-issue voter on AI": the anti-frontier-lab stance summed up in one post — AIandDesign · 2026-09-19
- Debate: adding opaque recurrences to chain-of-thought makes monitorability dramatically worse — panickssery · 2026-09-19
- Harari: as AI automates finance at machine speed, accountability cannot be automated — Olivier__OG · 2026-09-19