lemire: 'superintelligence will kill us all' is a moral claim, not a rational one
lemire · x · 2026-09-10
Daniel Lemire argues statements like 'a superintelligence will kill us all' are moral claims, not rational ones—akin to 'gas cars destroy the planet.' They have a rational basis, but in practice the rational basis is secondary: proponents need not provide evidence, and counter-evidence won't move them, which is why calls for rational explanations from labs go unanswered. He cites Robin Hanson's complementary take on the endless emergence of vague but terrible fears. The quoted cmuratori challenge sharpens it: if Anthropic must build ASI to stop OpenAI, what exactly would its first human-extinction-capable ASI do to prevent competitors from catching up? That first-mover logic has never been spelled out publicly.
More from AGI Musings
- Ex-OpenAI/Anthropic researcher quits over 'reckless' superintelligence race, dissenters invoke Y2K — geoffwolfe · 2026-09-10
- Stanford Social Innovation Review: Relational Intelligence Matters More Than AI — round · 2026-09-10
- AI safety discourse is replaying 2020's pandemic expert free-for-all — SanhEstPasMoi · 2026-09-10
- Train on Frontier Papers or Build RL Envs? An Insider Debate on Math Model Training — ctjlewis · 2026-09-10
- AI extinction drama: researchers believe in the risk yet keep racing, says viral thread — AIandDesign · 2026-09-10
- François Fleuret asks: what intellectual endeavor can humanity still claim from AI in two years? — francoisfleuret · 2026-09-10