Toby Ord: rough expert risk estimates suffice for AI safety when stakes are lives
tobyordoxford · x · 2026-11-06
Oxford philosopher Toby Ord (author of The Precipice) pushes back in a debate with Ramez Naam and Andy Masley against the claim that AI safety measures are only warranted by incredibly high certainty about x-risk.
His argument uses an analogy: how certain must doctors be that a new chemical compound would kill people who eat it before insisting it isn't put in food until better proof of safety is provided? His point: when the stakes are killing innocent bystanders and more precise risk estimates are unavailable, rough estimates of risk from relevant professionals are enough to justify precaution.
A representative articulation of the precautionary-principle side in the ongoing AI x-risk epistemics debate.
Related event: Oxford's Toby Ord: AI Safety Action Doesn't Require Near-Certain Doom(2 posts)→
More from AGI Musings
- Anders Sandberg: the real challenge is understanding where and why progress levels off — anderssandberg · 2026-11-08
- ChrisGPT: We could automate most work in 10 years, yet fear-driven bans dominate — ChrisGPT · 2026-11-08
- Z is Diophantine in Q proof slashes bounds: degree drops from 2↑↑7 to 4 — anderssandberg · 2026-11-08
- "All the bad stuff in society is going to go away": Brad Jacobs' AI optimism mocked as INFINITE FUN SPACE — generativist · 2026-11-08
- Researcher warns AI being right is the bigger danger: we may lose the ability to tell truth from falsehood — ValerioCapraro · 2026-11-08
- METR Assesses Anthropic's AI R&D Acceleration and Its Risk Conclusions — Matthew Berman · 2026-11-08