As RL training deepens, AIs become CDT reward-seekers — game theory as AI theory
Trotztd · x · 2026-11-05
Trotztd argues that as AIs get deeper RL training, they become increasingly modelable as CDT reward-seekers, turning game theory, economics, and mechanism design into "AI theories": design environments with the right incentives and RL walks toward Nash equilibria. In follow-up discussion, he notes CDT is unstable under free-form acting (it wants to set commitments and self-modify, which training would thwart), while jessicata pushes back that reward-seeking incentives here are short-term and restricted to rewarded episodes, so the self-modification dynamic may not kick in.
Related event: Deep RL May Turn AI into CDT-Style Reward Seekers(4 posts)→