Borretti's 'Possible Futures': mapping ASI outcomes, alignment and takeoff severity
zetalyrae · x · 2026-11-12
zetalyrae shares Fermín Koop-Sotelo's long-form essay "Possible Futures," a systematic taxonomy of ASI outcomes:
- Five key questions: whether ASI is reached or capabilities stay jagged; whether alignment is solvable; intent alignment vs. value alignment; how fatal misalignment is (paperclip maximizer vs. benign non-interference); singleton vs. pantheon vs. ecology; and full job automation in a no-ASI world.
- Misuse Doom scenario: AI ends the world by following operator orders—via strongly intent-aligned models, refusals-ablated open-source models, or safeguards circumvented by task decomposition. Biorisk is the nearest path: even a legitimate government-funded gain-of-function lab could accidentally trigger a pandemic.
- Framed around five branching questions with exhaustive scenario analysis; a high-substance AGI futures essay.
More from AGI Musings
- Turing Award winner Patterson: Robotaxis could shrink 280M US cars to 56M in 3.5 years — davidpattersonx · 2026-11-12
- AI Trials Observatory launches: living meta-analysis of 93 RCTs on AI's human impact — RobbWiller · 2026-11-12
- 'Not obviously useless' is a high bar in 2026: creator on standing out from AI slop — wordgrammer · 2026-11-12
- Lab employees refusing to work could be the fastest way to pause AI, argues researcher — jungofthewon · 2026-11-12
- AI is our greatest hope of escaping the shackles of evolution, argues DeryaTR_ — DeryaTR_ · 2026-11-12
- Blogger calls for 'frontier philosophy labs' to tackle AGI-era questions — round · 2026-11-12