Debating extinction vs catastrophic AI risk misses the point: the guardrails are the same
bryanrwalsh · x · 2026-10-27
Bryan Walsh cites a discussion where AI-safety skeptic Kate Brennan argues the biggest AI risk is AI being pushed "everywhere, into all of our public institutions and contracting," including use cases where it isn't secure or safe by design.
Walsh presses on where Brennan draws the line: healthcare/government cyberattacks count, but what about catastrophic infrastructure failures? His core point: debating whether AI poses "extinction-level" vs merely catastrophic risk mostly gets hung up on the term itself. Borrowing Derek Parfit's framing that 100% death is far worse than 99% death, he argues that in practice, once you're in the catastrophe zone, most of the useful guardrails — like independent review — are the same either way.
More from AGI Musings
- Vox: Anthropic should quit the AI race unless it can prove its safety strategy works — GarrisonLovely · 2026-10-27
- Stanford's 457-page AI Index: cost down ~30%/yr, top-model gap narrows to 5.4% — mdancho84 · 2026-10-27
- Observer Launches 'Spiral' Podcast Investigating How AI Chatbots Warp Minds — TheObserverUK · 2026-10-27
- AI Now Institute: bets on our AI future are being made by a handful of billionaire tech CEOs — AINowInstitute · 2026-10-27
- AI Text Is Flattening Human Style: TikTok Coins New Words, LLMs Revive Dormant Ones — BecauseCulture · 2026-10-27
- AI Agents That Move Money Now 9% of Crypto VC Pitches, Up From 1% in 2023 — LexSokolin · 2026-10-27