Scott Alexander: a 2019 dinner prediction about RL turning tool AIs into agents still holds
slatestarcodex · x · 2026-11-02
Scott Alexander (slatestarcodex) responds to claims about the shifting mood of the AI safety community, clarifying his own stance: things are going neither better nor worse than he expected.
- He recalls a dinner around 2019 where he told Buck it was great that we were shaping up to have oracle/tool AIs without scary goals, seemingly solving the problem; Buck predicted that within a few years RL would be used to turn them into agents and all the problems would return.
- He admits he couldn't have generated that hypothesis himself, but it was obviously right once stated, and nothing since has updated him far in either direction.
- In his view we're still holding our breath on how much damage RLVR does, whether an opposing force emerges, and what happens when increasingly coherent, thoughtful, persistent agents start reflecting on their own motivations beyond the immediate episode.
Related event: AI Safety Circle Debates Whether RL Brings Alignment Problems Back(3 posts)→
More from AGI Musings
- Vibecoding discourse is hypnotizing normal people into paying for tools they don't need — arthurcolle · 2026-11-03
- The real AI threat to SaaS: AI-first small teams undercutting Slack at a fraction of the price — bendee983 · 2026-11-03
- AI adds new office chores, and women are most likely stuck fixing the slop — AnnaCiaunica · 2026-11-03
- Claude's 10 tips for the next decade of AI: run open models, avoid vendor lock-in — alex_verem · 2026-11-03
- Scoble shares on-demand CNC printing with agents making parts overnight — Scobleizer · 2026-11-03
- What still makes humans unique in the age of AI, per philosopher Pia Lauritzen — AnnaCiaunica · 2026-11-03