If we keep scaling AI, can we reliably avoid catastrophe? Not right now
AndyMasley · x · 2026-09-05
Andy Masley argues that stripped of the word "alignment", the question of whether continued AI scaling will reliably avoid catastrophic behavior while pursuing normal goals currently answers "no". He worries people over-update on the observed safety of today's models and assume future models won't have general human-like reasoning abilities. He adds that despite unknown unknowns, the risk story is plausible enough to enter mainstream debate, noting catastrophe scale is capped by capabilities — which show no signs of stalling.
Related event: Researcher: Continued AI scaling cannot reliably avert disasters(3 posts)→
More from AGI Musings
- India needs an "AI defence force": experts warn of agent swarms and AI worms — ravi_iitm · 2026-10-23
- Anti-AI Slop Books: open tools to detect AI-generated books — TuhinChakr · 2026-10-23
- Founder predicts mass tech layoffs as 2-person AI teams replace big corp teams — kingslyj · 2026-10-23
- Social scientist: CS treats agentic development as a magic black box — RexDouglass · 2026-10-23
- Blogger slams Anthropic for wanting approved institutions to gate access to AI — VraserX · 2026-10-23
- Senior researcher rebuts 'one-person pretraining' fear: your fundamentals matter more, not less — RexDouglass · 2026-10-23