Ex-OpenAI policy chief Brundage: AI safety is sociotechnical; security should top AI policy

Miles Brundage, former OpenAI policy lead, published a series of long posts on 09-17 and a follow-up essay on 09-18 laying out his view of AI safety and policy priorities: AI safety is not a purely technical problem but a sociotechnical one with no one-shot solution; he argued that security—preventing AI technology from being stolen or tampered with—should be the top AI policy priority, and that security progress failing to keep pace with capability gains is the strongest argument for slowing down.

Confirmed

• Brundage argued that anti-regulation voices framing AI safety as a "purely technical problem" is a motte-and-bailey move: it makes the problem seem easy to solve and shields the industry from criticism.
• He analogized AI safety to complex sociotechnical systems like aviation, nuclear power, and healthcare: without constraints, under time pressure and competition, people cut corners, slack off, and fall into groupthink—technical fixes alone cannot solve this.
• He highlighted a "safety gains reinvestment" effect: safety headroom gets converted into larger pretraining runs, harder RL environments, more test-time compute, and looser internet access; there is no clear ceiling on AI capability, and no company will declare "we're safe enough" and stop.
• Discrete safety improvements are insufficient—gains are reinvested into new risks; even if a recipe for perfectly safe/aligned frontier models existed tomorrow (he stressed it does not), companies would still cut corners chasing the next tier under competition.
• Drawing on nuclear power: even with extensive design-specific safety regimes, there has never been a universal "solve it once" fix—AI likewise requires continuous iteration.
• He proposed a test: any AI governance theory that cannot survive historical backtesting against dangerous systems is not credible; a single bad act is not doomsday, but without guardrails it can escalate.
• On 09-18 he wrote that security against theft and tampering should be the top AI policy priority, explained his shift toward the issue, said he would work on it if he had more to contribute, and noted with regret that security progress lags capability growth—perhaps the strongest argument for slowing down.

Why it matters

• This is a systematic argument from a former OpenAI policy chief, directly rebutting the "AI safety is merely technical, no regulation needed" narrative and offering a conceptual framework for AI governance debates.
• The "safety gains reinvestment" and "no capability ceiling" arguments imply safety problems accumulate as capabilities scale, with direct implications for policymaking and corporate governance.
• His elevation of anti-theft, anti-tampering security to top policy priority, and his frank admission that security lags capability, give the "slow down" camp an argument from a former insider at a leading lab.

2026-09-17 ~ 2026-09-18 · 9 related posts

Full story(2 episodes)→

• Episode 1: Ex-OpenAI policy chief Brundage pivots to frontier AI auditing(2026-09-16, 9 posts)
• Episode 2: Ex-OpenAI policy chief Brundage: AI safety is sociotechnical; security should top AI policy(2026-09-17, 9 posts)

• [source] Brundage: Framing AI Safety as Purely Technical Is an Anti-Regulation Move — Miles_Brundage · 2026-09-17
• Brundage: Even Perfect AI Alignment Wouldn't Stop Competitive Corner-Cutting — Miles_Brundage · 2026-09-17
• Brundage: AI Safety Has No Once-and-For-All Solution, Just Like Nuclear — Miles_Brundage · 2026-09-17
• Brundage: AI Safety Is the Same Socio-Technical Problem as Aviation, Nuclear — Miles_Brundage · 2026-09-17
• Miles Brundage: Discrete AI Safety Gains Just Get Reinvested Into New Risks — Miles_Brundage · 2026-09-17
• [source] Brundage: No Clear Ceiling Where AI Companies Will Say "We're Safe Now" — Miles_Brundage · 2026-09-17
• Miles Brundage: Don't Trust AI Regulation Theories That Fail Historical Backtests — Miles_Brundage · 2026-09-17
• [source] Miles Brundage: Preventing AI Theft and Tampering Is the Top Policy Priority — Miles_Brundage · 2026-09-18
• Brundage: AI Security Progress Lags Capability Gains—the Strongest Case for Slowdown — Miles_Brundage · 2026-09-18