Even the Safest AI Leads to a Future This Researcher Finds Deeply Unsettling
mayfer · x · 2026-09-15
Author mayfer argues that after watching AI's first three years unfold, even the safest AI leads to a future he finds deeply unsettling.
He walks through which classic risks he does not worry about:
- Self-replicating robots: manufacturing is too entangled with the physical, legal and funding world to go unnoticed early.
- Nano self-replicators consuming Earth: you can't just "think" your way into that tech tree.
- Pure digital hacking: easy to recover from total loss of digital records, whereas attacks on physical systems are harder to undo (thread continues).
Related event: AI risk essay: cultural upheaval, not nanobots, is the real worry(2 posts)→
More from AGI Musings
- AI risk researcher: "I can't imagine extinction scenarios" isn't a rebuttal — davidmanheim · 2026-09-15
- Dario Amodei calls AI progress a 'warning sign' and says we need to slow down — tekbog · 2026-09-15
- DeepSeek kernel engineer frames open-source AI work as stopping Anthropic dominance — tommos · 2026-09-15
- What Amodei's call for an AI pause gets wrong: self-interested oversight — Gloomy_Register_2341 · 2026-09-15
- Why there's no AI spam flood yet: 100x the cost of mail merge for only 2-10x the CTR — paul_cal · 2026-09-15
- Insiders push back on AI-virus threat models: ordering viral fragments as a rando gets you reported to the FBI — basedjensen · 2026-09-15