Dario's 2016 AI safety paper maps eerily well onto today's LLM agents
terryyuezhuo · x · 2026-11-21
Netizen terryyuezhuo dug up Dario Amodei's 2016 classic "Concrete Problems in AI Safety" (arXiv:1606.06565, with Chris Olah, Paul Christiano, John Schulman et al.), noting that swapping the paper's "cleaning robot" for "LLM agent" makes all five problems map directly onto today's systems: avoiding negative side effects, avoiding reward hacking, scalable oversight, safe exploration, and robustness to distributional shift — evidence Amodei has been thinking about agent safety for nearly a decade.
Related event: Dario's 2016 AI Safety Paper Still Maps Neatly Onto Today's LLM Agents(2 posts)→
More from AGI Musings
- AI writes 90% of code but plateaus at ~50% of online articles — bendee983 · 2026-11-21
- Jack Dorsey says Block is 'a lot flatter' with 3 core roles after AI-driven restructuring — shivsingh · 2026-11-21
- Superintelligence could solve cancer treatments theoretically before wet labs — Dr_Singularity · 2026-11-21
- Yang You's 'Large Partial Derivative Theory': pick research where your marginal contribution is biggest — YangYou1991 · 2026-11-21
- Looking back: OpenAI reasoning model autonomously disproved Erdős's unit-distance conjecture — aran_nayebi · 2026-11-21
- Government-funded kids channel looks AI-generated, sparking animator fears — justalexoki · 2026-11-21