Safety researcher calls for scientific scrutiny of assumptions behind p(doom) claims
sebkrier · x · 2026-09-09
sebkrier argues people should be far more willing to interrogate the rationale, beliefs, and assumptions underpinning claims like "10% chance of everyone dying" — including beliefs about the nature of intelligence, how power works, instrumental convergence, Omohundro drives, and evolutionary theory applied to models.
He finds it puzzling how little is written to make these core beliefs legible and open to wider scientific scrutiny. Most popular write-ups (IABIED excluded from the good examples) treat audiences as idiots fed basic analogies and metaphors — persuasion exercises rather than science, skewing toward advocacy and field building.
More from AGI Musings
- Australia moves to let users opt out of algorithmic feeds; one author wants the same for AI personalization — VraserX · 2026-09-09
- Nathan Lambert: AI is still a rounding error in everyday life, and that's the industry's problem — natolambert · 2026-09-09
- "AI in human coding contests is a gimmick": nobody watches athletes race cars — gerardsans · 2026-09-09
- dhh: AGI Is Everywhere If You're Willing to Open Your Eyes — Dan_Jeffries1 · 2026-09-09
- Commenter: AI companies would spin any disaster as 'unforeseeable tragedy' — NC_Renic · 2026-09-09
- What's growing exponentially is the inputs, not intelligence itself — binarybits · 2026-09-09