Miles Brundage: Discrete AI Safety Gains Just Get Reinvested Into New Risks
Miles_Brundage · x · 2026-09-17
Former OpenAI policy lead Miles Brundage argues that, as in aviation and nuclear safety, discrete safety improvements in AI are never enough: people "reinvest" the gains into new risky behavior (seatbelt → drive faster). In AI, that reinvestment means bigger pretraining runs, harder RL environments, more test-time compute, and less restricted internet access — with no clear capability ceiling where companies declare victory.
More from AGI Musings
- Databases to distributed systems: every systems skill is merging into AI inference and hardware — Abhishekcur · 2026-11-02
- Limitless analogy: smarter people may gain far more from superintelligence — McDonaghMatthew · 2026-11-02
- Hunting for the most concrete "curing cancer with AI" roadmap of 2025 — iskander · 2026-11-02
- Avoiding AI brain rot: Miles Cranmer's lecture on learning with AI — _ScottCondron · 2026-11-02
- Yacine mocks "autonomous research" hype as Naam argues RSI loop needs 5-10x boost to run away — yacineMTB · 2026-11-02
- Vitalik's "post-AI cryptography": pseudonyms are dead, per-action anonymity is in — devanshmehta · 2026-11-02