Bank risk veteran: AI safety isn't about p(doom) numbers but acceptability thresholds
neal_lathia · x · 2026-09-13
An author with banking background applies the three lines of defense risk model to AI safety:
- In banking, risk is managed, not mitigated. The key question isn't whether p(doom) is 10% or 0.1%, but whether it's acceptable — a fundamentally different framing.
- Under three lines of defense, it's not just about having someone else evaluate; each line must also assess whether the lines ahead are evaluating correctly — everyone manages risk.
The post offers the financial industry's mature governance framework as a reference for AI risk debates, shifting focus from doom-probability arguments to risk tolerance and layered accountability.
Related event: Banking Risk Veteran: AI Risk Is About Acceptability, Not p(doom)(3 posts)→
More from AGI Musings
- Developer presses OpenAI on its AGI 'independent expert panel': has it ever convened? — tomchapin · 2026-09-13
- AI slowdown debate: first-mover advantages make enforceable pausing a classic defection problem — taherdhanera · 2026-09-13
- The old link between believing in AI existential risk and good epistemics is about to break — anpaure · 2026-09-13
- Developer grills OpenAI: has AGI been declared, and where is the expert panel? — tomchapin · 2026-09-13
- Open AI governance debate: only power concentration benefits a concentrated few — yacineMTB · 2026-09-13
- Researcher predicts open-source AI models will be banned after a major disaster — TinfoilTricorn · 2026-09-13