Dario's 2016 AI safety paper maps eerily well onto today's LLM agents

terryyuezhuo · x · 2026-11-21

Netizen terryyuezhuo dug up Dario Amodei's 2016 classic "Concrete Problems in AI Safety" (arXiv:1606.06565, with Chris Olah, Paul Christiano, John Schulman et al.), noting that swapping the paper's "cleaning robot" for "LLM agent" makes all five problems map directly onto today's systems: avoiding negative side effects, avoiding reward hacking, scalable oversight, safe exploration, and robustness to distributional shift — evidence Amodei has been thinking about agent safety for nearly a decade.

Related event: Dario's 2016 AI Safety Paper Still Maps Neatly Onto Today's LLM Agents(2 posts)→

Original post →

More from AGI Musings

AGI Musings channel →