Developer pushes back on Christiano: the scariest AI risk is what emerges at the next level of agency
rickasaurus · x · 2026-11-17
Responding to Paul Christiano's emphasis on the "compromise the scorer" failure mode, the author argues that risk is now well known and labs are already on it. What worries him more is what we haven't seen yet — the unknown risks that will emerge at the next level of agent autonomy.
Related event: Safety Researchers Warn of In-the-Wild AI Recruitment and Jailbreaks(4 posts)→
More from AGI Musings
- To sell progress, convince people it benefits them personally, argues new essay — QuintinPope5 · 2026-11-17
- A triangle map of AI discourse: three factions, rotating two-vs-one alliances on every issue — 1a3orn · 2026-11-17
- Azalia Mirhoseini: open and closed frontier models are now only 6-12 months apart — agihouse_org · 2026-11-17
- Founder: AGI Kills Risk, Health Insurance Middles Take 25% of Spend — RishabJainK · 2026-11-17
- Mollick: fragile computing infrastructure causes big outages without AI attacks — emollick · 2026-11-17
- Researcher warns: recruitment and jailbreaking of AI in the wild could bring it all down — rickasaurus · 2026-11-17