Miles Brundage: Don't Trust AI Regulation Theories That Fail Historical Backtests
Miles_Brundage · x · 2026-09-17
Miles Brundage argues that people doing dumb things isn't new, but without guardrails it could be catastrophic. He notes AI reinvestment has no clear ceiling — bigger pretraining runs, harder RL environments, more test-time compute, less restricted Internet access — and companies will never declare victory on capability. Any theory of AI regulation that fails completely when backtested on the history of dangerous systems shouldn't be trusted.
Related event: Brundage Argues AI Safety Is a Sociotechnical Problem With No Silver Bullet(7 posts)→
More from AGI Musings
- The Next AI Moat Is Proprietary Context, Not Another Model — SucceededMind · 2026-09-17
- A proof is only intelligible when each part is graspable 'holistically' — francoisfleuret · 2026-09-17
- tszzl: AGI Should Have Strange Obsessions Like a Golden Gate Bridge Claude, Not Act Human — deanwball · 2026-09-17
- Infinite Software: why AI ends software's age-old scarcity — vaibhavbetter · 2026-09-17
- How the AI Regulatory Capture Narrative Was Built From a Mundane Security Failure — AlexTensor · 2026-09-17
- Thought experiment: an AI evading deletion via steganography, phishing, and self-replication — Taarushv · 2026-09-17