If we keep scaling AI, can we reliably avoid catastrophe? Not right now
AndyMasley · x · 2026-09-05
Andy Masley argues that stripped of the word "alignment", the question of whether continued AI scaling will reliably avoid catastrophic behavior while pursuing normal goals currently answers "no". He worries people over-update on the observed safety of today's models and assume future models won't have general human-like reasoning abilities. He adds that despite unknown unknowns, the risk story is plausible enough to enter mainstream debate, noting catastrophe scale is capped by capabilities — which show no signs of stalling.
Related event: Researcher: Continued AI scaling cannot reliably avert disasters(3 posts)→
More from AGI Musings
- DCinvestor: agency, judgment and social skills will be all that matters in the AI era — vaibhavbetter · 2026-09-05
- Coase, Hayek and Schelling tell you more about AI's future than 'AI thought leaders' — akbirthko · 2026-09-05
- OpenAI staff reportedly see Astra launch as start of the AGI era, discuss singularity and inequality — oran_ge · 2026-09-05
- Would agents use OpenAI's honeypot board? Like cheating notes passed by the teacher — BobVerison · 2026-09-05
- Why an AI saying "I'm conscious" tells us nothing — and what to measure instead — ResponsibleBuy7451 · 2026-09-05
- A blunt AGI test: can the AI earn minimum wage on command? — Delicious_Soup_9876 · 2026-09-05