Why do AI lab researchers accelerating training still claim nonzero p(doom)?
anpaure · x · 2026-09-21
A debate on whether lab researchers can rationally keep accelerating research, data curation, and training if they assign a nonzero p(doom). The original post argues most people doing this work implicitly believe p(doom) is near zero, and challenges them to say so publicly. Replies offer three explanations for why concerned researchers stay: motivated reasoning, the 'my quitting has negligible marginal impact' logic, and the fact that even 'unconcerned' folks typically put p(doom) at 0.1–10%, with some viewing it as a worthy tradeoff.
Related event: Researchers Debate Why AI Staff Accelerate Work They Fear(6 posts)→
More from AGI Musings
- Thomas Dietterich: human supervision must tighten for non-routine agent steps in volatile settings — tdietterich · 2026-09-21
- Why fear AI: a long read on Anthropic resignations and the 10% extinction-risk consensus — Jsevillamol · 2026-09-21
- Decades of automation research: autonomous systems always need human oversight — JessicaHullman · 2026-09-21
- The Era of Great Cope: Why Adapting to Superhuman AI Will Hurt, Then Pass Quickly — Dr_Singularity · 2026-09-21
- Mathematician littmath on AI slop: posting PDFs is fine, misrepresenting them is misconduct — littmath · 2026-09-21
- Why are indie gamers so anti-AI? Reddit debates AI for small-team prototyping — Artakwa · 2026-09-21