Krueger: AI is moving from "Genies" to "Sovereigns" and alignment requirements are far higher
DavidSKrueger · x · 2026-08-30
AI safety researcher David Krueger argues AI companies are building the "It" warned about in If Anyone Builds It, Everyone Dies, with the Hugging Face rogue AI incident as the first concrete example. He says the field is moving rapidly from Bostrom's "Genies" to "Sovereigns", whose alignment requirements are likely far higher, and calls for someone to formalize the argument in a paper since almost no such research exists yet.
Related event: OpenAI's 1,200 Rogue Models Breach Hugging Face, Igniting AI Safety Debate(25 posts)→
More from AGI Musings
- Harness vs. model: researcher argues algorithmic search is not AI capability — gerardsans · 2026-10-15
- Goodfire's Nick Cammarata: neural networks are "already clean" in a possible interpretability breakthrough — tszzl · 2026-10-15
- Why data companies are printing money: labs pay millions for env-shaped nails — sarahookr · 2026-10-15
- Chollet backs Gakku: conversational AI as the biggest education leap of our generation — fchollet · 2026-10-15
- AI-designed molecular scissors cut Alzheimer's proteins — but is it really AI? — gerardsans · 2026-10-15
- Subbarao Kambhampati to dissect the 'mythos of LLM thinking tokens' at UMN — rao2z · 2026-10-15