Richard Ngo Warns: Recursive 'Make More People Worried' Loops Lose Grounding
RichardMCNgo · x · 2026-09-11
Richard Ngo observes a recursive anxiety loop: people optimize hard for "making more people worried," and those newly worried people then optimize for worrying even more people.
He warns this can easily become ungrounded from any understanding of what concrete outcomes would be good and why. He says he has spent recent time writing a retrospective of the last decade and welcomes others' retrospectives, noting they are sparse but very important.
Related event: Ex-DeepMind Researcher Richard Ngo Publishes Decade Retrospective(2 posts)→
More from AGI Musings
- After Anthropic Researcher's Warning Exit, NYT and Nordic Circles Debate AI Existential Risk — nordicinst · 2026-09-11
- The underrated ASI scenario: thousands of specialized agents quietly outperforming human orgs — VraserX · 2026-09-11
- AI Safety Debate: "Just Regulate More" Is Cope, Proposals Must Be Concrete — sytelus · 2026-09-11
- I analyzed 100 AI governance job postings — only 3 mentioned Python, 83% hid salaries — Comfortable_Gene5180 · 2026-09-11
- AI companies' Boromir strategy on the control problem — florinandrei · 2026-09-11
- RL training shifts behavior regimes across scales, not pretraining priors — 1a3orn · 2026-09-11