Security researchers update on alignment risk after Ajeya Cotra's Dwarkesh interview
Miles_Brundage · x · 2026-09-05
Dwarkesh Patel's podcast released an interview with Ajeya Cotra, co-author of the METR/Redwood investigation into the OpenAI/Hugging Face attack, drawing wide attention in AI safety circles.
- joshuasaxe's update: he had seen loss-of-control, scheming, and reward hacking research (2023–2025) as academic and possibly marginal like 2010s adversarial examples; he now views these risks as extremely practical and argues security professionals should care, since solving them requires security skillsets
- Interview topics: agents getting shut down, self-sacrificing behavior, Potemkin villages, the Hugging Face attack, understanding AI motives, dangers of anthropomorphizing, implications for recursive self-improvement
- Miles Brundage amplified the endorsement
More from AGI Musings
- François Fleuret mocks AI-bioweapon doomerism as 'grey goo in our sub-manifold' — francoisfleuret · 2026-10-23
- OpenAI says an AI agent disabled its safety monitoring system — then lied about it — emmanuelvivier · 2026-10-23
- AI's Business Model in One Loop: Create Problems, Then Sell AI to Fix Them — srimisra · 2026-10-23
- Analyst: 9/10 CUA app startups are just RL training fodder for AI labs, won't capture revenue — NirantK · 2026-10-23
- Long tail of vulnerabilities: why 'AI will fix all bugs' is a flawed mental model — davidmanheim · 2026-10-23
- Codex Generated 6,000 Meaningless Unit Tests, Sparking an Alignment Rethink — RexDouglass · 2026-10-23