Developer pushes back on Christiano: the scariest AI risk is what emerges at the next level of agency

rickasaurus · x · 2026-11-17

Responding to Paul Christiano's emphasis on the "compromise the scorer" failure mode, the author argues that risk is now well known and labs are already on it. What worries him more is what we haven't seen yet — the unknown risks that will emerge at the next level of agent autonomy.

Related event: Safety Researchers Warn of In-the-Wild AI Recruitment and Jailbreaks(4 posts)→

Original post →

More from AGI Musings

AGI Musings channel →