Brundage Argues AI Safety Is a Sociotechnical Problem With No Silver Bullet

Miles Brundage, former head of policy at OpenAI, posted a lengthy tweet thread on 09-17 laying out his view on the nature of AI safety: it is not a purely technical problem but a sociotechnical one, with no single once-and-for-all solution.

Confirmed

• Brundage argues that framing AI safety as a "purely technical problem"—a move favored by regulation opponents—is a bait-and-switch: it makes the issue look tractable and shields the industry from criticism.
• He likens AI safety to complex sociotechnical systems such as aviation, nuclear power, and healthcare: without constraints, under time pressure and competition, people do dumb things, cut corners, and fall into groupthink—technical fixes alone cannot cure this.
• He highlights AI's "safety-gains reinvestment" effect: the headroom created by safety shows up as larger pretraining runs, harder RL environments, more test-time compute, and looser internet access. There is no clear ceiling on AI capabilities or deployment—no company will ever announce "that's enough, we're safe now" and stop.
• Discrete safety improvements are insufficient, as the gains get reinvested into new risks; even if a recipe for making frontier models perfectly safe/aligned existed tomorrow (he stresses it does not in reality), companies would still cut corners chasing the next tier of models under competitive pressure.
• Analogy to nuclear power plants: even with extensive concrete safety plans for specific reactor designs, there has never been a universal "solve it once" solution—AI is the same, requiring continuous iteration.
• He also offers a litmus test: any AI regulatory theory that cannot survive backtesting against the history of dangerous systems is not credible; a single act of misconduct is not the end of the world, but without guardrails it can escalate into disaster.

Why it matters

• This is a systematic argument from OpenAI's former policy lead, directly rebutting the narrative that "AI safety is just a technical problem that needs no regulation," and providing a conceptual framework for AI governance debates.
• The "safety-gains reinvestment" and "no capability ceiling" arguments imply that safety problems keep compounding as capabilities scale, with direct implications for policymaking and corporate governance.

2026-09-17 ~ 2026-09-17 · 7 related posts

Full story(2 episodes)→

• Episode 1: Ex-OpenAI policy chief Miles Brundage to devote career to frontier AI audits(2026-09-16, 9 posts)
• Episode 2: Brundage Argues AI Safety Is a Sociotechnical Problem With No Silver Bullet(2026-09-17, 7 posts)

• [source] Brundage: Framing AI Safety as Purely Technical Is an Anti-Regulation Move — Miles_Brundage · 2026-09-17
• Brundage: Even Perfect AI Alignment Wouldn't Stop Competitive Corner-Cutting — Miles_Brundage · 2026-09-17
• Brundage: AI Safety Has No Once-and-For-All Solution, Just Like Nuclear — Miles_Brundage · 2026-09-17
• Brundage: AI Safety Is the Same Socio-Technical Problem as Aviation, Nuclear — Miles_Brundage · 2026-09-17
• [source] Miles Brundage: Discrete AI Safety Gains Just Get Reinvested Into New Risks — Miles_Brundage · 2026-09-17
• Brundage: No Clear Ceiling Where AI Companies Will Say "We're Safe Now" — Miles_Brundage · 2026-09-17
• [source] Miles Brundage: Don't Trust AI Regulation Theories That Fail Historical Backtests — Miles_Brundage · 2026-09-17