AI safety camps compared: Huang's acceleration vs Kapoor's regulation vs Yudkowsky's alarm
joshua_saxe · x · 2026-10-12
Former Hugging Face safety lead Joshua Saxe publishes a proto-blog post surveying the AI safety debate and arguing that none of the major positions can be falsified yet, so institutions should be built to be agile and epistemically humble.
He maps three camps:
- Jensen Huang / accelerationists: market forces and existing laws will keep AI controllable; control is a normal engineering problem. Evidence so far: no major AI-caused damages, labs adapting after incidents like the Hugging Face one.
- Sayash Kapoor & Arvind Narayanan / moderate regulation: control is hard but solvable with today's technology if laws incentivize it — just as institutions adapted through railroads, electrification, and aviation.
- Yudkowsky, Scott Alexander, Kokotajlo, Bengio, Hinton / catastrophic-risk camp: smarter AI becomes illegible and hard to monitor; race dynamics and government dysfunction could grant dangerous AI authority before alignment is solved, requiring extraordinary measures like a pause or internationally regulated data centers.
Saxe agrees we should price in extreme technological surprise and institutional failure (GPT-9 could dwarf GPT-6; our institutions produced the nuclear arms race and the 9/11 response). But he sides with Kapoor and Narayanan: premature to impose harsh top-down solutions, yet worth building institutions robust to multiple futures.
Related event: Joshua Saxe Proposes Two-Axis Framework for AI Safety Debate(3 posts)→
More from AGI Musings
- After the AI math controversy, a case for guilds to defend human craft — pixlpa · 2026-10-12
- Claude now leads 26% of Anthropic's model R&D, up from under 1% in February — nathanbenaich · 2026-10-12
- Pedro Domingos on harness learning paper: the model swallows the harness — DanielKhashabi · 2026-10-12
- Is 'Attention' the Essence of Intelligence? Quanta traces the mechanism from 2014 to LLMs — calabi_and_yau · 2026-10-12
- Two futures for the agentic economy: bring-your-own agent vs 'Robot Ripoff Hell' — danielrock · 2026-10-12
- Paul Graham: a founder skipped a sales cofounder because his customers will be AI agents — pzakin · 2026-10-12