Miles Brundage: Discrete AI Safety Gains Just Get Reinvested Into New Risks
Miles_Brundage · x · 2026-09-17
Former OpenAI policy lead Miles Brundage argues that, as in aviation and nuclear safety, discrete safety improvements in AI are never enough: people "reinvest" the gains into new risky behavior (seatbelt → drive faster). In AI, that reinvestment means bigger pretraining runs, harder RL environments, more test-time compute, and less restricted internet access — with no clear capability ceiling where companies declare victory.
More from AGI Musings
- AI safety researcher warns against regulation that looks like regulatory capture — DavidSKrueger · 2026-09-17
- Cambridge's David Krueger flags contradiction in US AI race policy — DavidSKrueger · 2026-09-17
- Reviewing AI code through Steve Jobs' lens: unseen internals deserve beauty too — sergeykarayev · 2026-09-17
- An economist's take: AI agent incidents are decades-old emergence, just with real-world stakes — Genzinvestor16180339 · 2026-09-17
- Ex-OpenAI researcher puts AI catastrophe odds at 70%, mocked online for doomer math — DeryaTR_ · 2026-09-17
- Cantos founder: deep tech progress comes from layered sustaining innovations, not one breakthrough — dr_alphalyrae · 2026-09-17