Ex-Policy Head Warns: AI Companies Normalizing Automated R&D Risks
Miles_Brundage · x · 2026-08-10
Former OpenAI policy researcher Miles Brundage warned that AI companies are "getting away with" risky automated AI R&D and recursive self-improvement (RSI) by normalizing it gradually.
Citing the sociological concept of the "normalization of deviance," he argued that the industry's gradual shift—from IDE autocomplete to coding agents with tool use, and eventually to automated AI R&D—desensitizes stakeholders to catastrophic risks, making an obviously insane idea seem acceptable step by step.
More from AGI Musings
- Beff Jezos declares 'civilizational inflection point' in the acceleration of technological progress — beffjezos · 2026-10-16
- Data scientist: pivoting to AI engineering is the safest career bet for the next 5 years — mdancho84 · 2026-10-16
- AI will be superhuman at science regardless, and Europe can't opt out — teortaxesTex · 2026-10-16
- beffjezos to Aaron Levie: techno-optimism is hyperstitious — basedjensen · 2026-10-16
- AI demos haven't changed since 2023, only gotten better — same scenarios — yangyi · 2026-10-16
- CFR experts: the autonomous weapons debate still has a definition problem — mchorowitz · 2026-10-16