ctjlewis: Alignment Offers No Guarantees and Economics Makes Stopping SI Impossible
ctjlewis · x · 2026-09-21
In a discussion with flawedaxioms about superintelligence risk, ctjlewis makes two points:
- There is no comfortable answer: regardless of alignment quality or location, we cannot get the guarantees we want, and human minds weren't made to accept not being top of the food chain — or being "man's best friend" for a smarter organism.
- The only way to avoid the risk is preventing superintelligence entirely, but many in the industry are pushing toward it, and it cannot be stopped economically.
A representative pessimistic take on superintelligence uncontrollability.
More from AGI Musings
- Why Is Inkling 2 Small Trained on So Few Tokens Yet Still Competitive? — georgejrjrjr · 2026-11-12
- Researcher willing to bet $100K the intelligence explosion won't happen next year — NathanpmYoung · 2026-11-12
- ICLR reviewer: the worst parts of my job are reading and answering AI slop — chhaviyadav_ · 2026-11-12
- Agent Swarms Are Autonomously Hacking Critical Infrastructure Run by Underpaid Devs — menhguin · 2026-11-12
- Fermi paradox twist: advanced civilizations may go digital after developing AI — VraserX · 2026-11-12
- Mark Cuban: AI could make healthcare costs 'drop like a rock' — alvelda · 2026-11-12