AI x-risk debate: the real danger is people with models, not the models
arian_ghashghai · x · 2026-09-10
Responding to a question about precisely what an AI-caused extinction scenario would look like, arianghashghai argues that fears about "the models" bringing chaos and destruction are really fears about what people with models will do.
- He frames ASI as the end stage of Enlightenment rationalism, noting Enlightenment philosophers saw humans in the state of nature as "wolves capable of reason" needing strong systems for restraint
- If individuals can gain tools that temporarily usurp those systems, our own prevailing philosophy suggests we should assume self-serving intent
- He traces the idea beyond the Enlightenment to Machiavelli's The Prince
More from AGI Musings
- Lab insiders reportedly quote ~10% AI doom publicly but believe 30-90% privately — jeremiecharris · 2026-09-10
- Most non-tech users just want ChatGPT as search, health advice and email drafts — bendee983 · 2026-09-10
- Paul Christiano joins OpenAI board as AI risk hits its 'March 2020' moment — NathanpmYoung · 2026-09-10
- Beff Jezos: 90% sure Anthropic will 'synthesize bioweapons' to spur regulatory capture — AIFlow_ML · 2026-09-10
- A dung beetle and an ant score alike on benchmarks, yet differ vastly in potential — sebkrier · 2026-09-10
- T.J. Clark argues Paleolithic cave art was a counter-move against language's dominance of thought — jjvincent · 2026-09-10