Seb Krier lays out his AI safety views: misuse and institutional failure over scheming AIs
sebkrier · x · 2026-11-07
Seb Krier shares a tl;dr of his AI safety views:
- Threat models: (a) non-state actors asymmetrically exploiting cheaper technologies over time (citing Bobbitt's Terror and Consent); (b) state overreach and creeping authoritarianism paved with good intentions. Misuse, bad engineering, and institutional failure matter more than alien 'scheming'.
- Safety and capabilities are intertwined: you need powerful systems for monitoring, reliability, and societal defenses; unsafe systems are commercially useless liabilities.
- Inner alignment alone is insufficient: abstract alignment is hard to specify and principal-agent problems loom — engineer system-level boundaries instead; ASI is not a joker card.
- Automating AI R&D is harder than claimed: gains stay tethered to bottlenecks, deployment feedback, costs, and human normative direction; more delegation ≠ humans out of the loop.
- Anti-fatalism: competitive pressure won't necessarily reduce humans to pets; automating execution pushes humans up the abstraction ladder toward governance — but liberal democracy should be actively protected.
More from AGI Musings
- Podcast uses Hindu karma concept to frame shared responsibility for AI harms — krvarshney · 2026-11-07
- TBPN Interview: AI in Drug Discovery, GLP-1s, and China's Biotech Rise — jordihays · 2026-11-07
- Has anyone notable from the Pause AI movement flipped? ChrisGPT asks why — ChrisGPT · 2026-11-07
- NYT: OpenAI breakthrough challenges the Exponential Time Hypothesis, shaking a CS tenet — fortnow · 2026-11-07
- DREADDs: a new drug modality that works in humans, with a clear path to AI-driven design — anshulkundaje · 2026-11-07
- Muse rolls out voice avatars as designers slam its 'cute' mascot for weaponizing empathy — joonasvirtanen · 2026-11-07