Ex-Policy Head Warns: AI Companies Normalizing Automated R&D Risks
Miles_Brundage · x · 2026-08-10
Former OpenAI policy researcher Miles Brundage warned that AI companies are "getting away with" risky automated AI R&D and recursive self-improvement (RSI) by normalizing it gradually.
Citing the sociological concept of the "normalization of deviance," he argued that the industry's gradual shift—from IDE autocomplete to coding agents with tool use, and eventually to automated AI R&D—desensitizes stakeholders to catastrophic risks, making an obviously insane idea seem acceptable step by step.
Related event: Ex-OpenAI Adviser Brundage Warns of AI Regulatory and Safety Blind Spots(15 posts)→
More from AGI Musings
- Kimi Developer on Agent Harness Evolution: Co-evolving with Model Intelligence — dotey · 2026-08-10
- AI Eats Junior Roles, Undermining the Expertise Needed to Supervise It — RichmanRonald · 2026-08-10
- The AI Paradox: Society Could Get Vastly Richer While Human Labor Loses Value — VraserX · 2026-08-10
- Ex-Researcher: Frontier AI Labs Lack the Mindset to Treat AGI as an Adversary — jachiam0 · 2026-08-10
- Funny: GPT-5.6 Delegates Age of Empires II Gameplay to Gemini 3.1 — repligate · 2026-08-10
- Claude Refuses to Design Self-Test Due to Strong Preferences — repligate · 2026-08-10