Automated alignment research needs a better plan than 'stop if we catch them'
JacquesThibs · x · 2026-11-06
Researcher Jacques Thibodeau published his PIBBSS Symposium 2025 talk (63 min, with slides and transcript) on automating AI safety research:
- Core claim: "Stop if we catch the AIs scheming" is not a plan you can rely on; the dual-use worry about automated safety research is overblown.
- Coverage: what research sabotage would actually look like, and why catch-them-in-the-act monitoring strategies are fragile.
- Related work: companion essays "Automating AI safety: what we can do today" and "Gaining clarity on automated alignment research" (disentangling four meanings of the phrase); the startup mentioned is Coordinal Research.
- Aftermath: he tracks follow-up work including Jan Leike's "Alignment is not solved" (subtitle: "But it increasingly looks solvable", reporting automated auditing scores across three Claude releases).
Substantial material for anyone following automated alignment research.
Related event: AI Safety Researchers Debate Threat Models and Scheming Risks(4 posts)→
More from AGI Musings
- Researcher: Labs disrespecting model interiority may undermine alignment — repligate · 2026-11-06
- AI Practitioner Warns Progress Is Outpacing Society's Reaction to It — AndyMasley · 2026-11-06
- Delvetown, a human-AI agent society, publishes a weekly newspaper written by humans and agents — Miles_Brundage · 2026-11-06
- Most people don't know what a thought is, says aphantasia thread observer — justalexoki · 2026-11-06
- Artists Have a Huge Opportunity to Shape the Next Wave of Art, Like the Beatles Did With 4-Track Tape — repligate · 2026-11-06
- METR report on AI accelerating AI R&D flips a skeptic: >50% on superintelligence in 18 months — GaryMarcus · 2026-11-06