Why do AI lab researchers accelerating training still claim nonzero p(doom)?
anpaure · x · 2026-09-21
A debate on whether lab researchers can rationally keep accelerating research, data curation, and training if they assign a nonzero p(doom). The original post argues most people doing this work implicitly believe p(doom) is near zero, and challenges them to say so publicly. Replies offer three explanations for why concerned researchers stay: motivated reasoning, the 'my quitting has negligible marginal impact' logic, and the fact that even 'unconcerned' folks typically put p(doom) at 0.1–10%, with some viewing it as a worthy tradeoff.
Related event: The p(doom) Debate: Do Lab Researchers Really Believe in Extinction Risk(8 posts)→
More from AGI Musings
- Agents' bottleneck isn't intelligence but goal commitment — and that's good for alignment — mimi10v3 · 2026-11-12
- Google Cloud survey: 78% say legacy enterprise vendors may not lead the AI era — rseroter · 2026-11-12
- Safety researchers blast OpenAI/Anthropic plan: automating alignment research with possibly misaligned AI — trevposts · 2026-11-12
- NYT essay: resisting Big Tech and AI takes collective action, not willpower — TuhinChakr · 2026-11-12
- Ethan Mollick: 'talk to your documents' RAG solutions already feel dated — emollick · 2026-11-12
- Anthropic's pitch hasn't changed in 5 years: 'AI is winner-take-all, we will build god' — menhguin · 2026-11-12