Thread Uses REINFORCE Policy Gradients to Explain RL's CDT/Nash Bias
jessi_cata · x · 2026-11-06
Citing a paper by Bell et al., this thread distinguishes two ways an agent's score can depend on its policy: environments depending on the policy directly (e.g., Omega), and dependence via selected actions. RL only assigns credit for the second kind. Analyzing REINFORCE gradients—where sample weights are identical and only sampling noise differs—the author explains the earlier claim that RL converges toward CDT/Nash-equilibrium behavior.
Related event: Deep RL May Push AI Toward CDT Reward-Seeking, Game Theory as AI Theory(6 posts)→
More from AGI Musings
- Commentary: AI CEOs have sucked at messaging why AI is good — AIandDesign · 2026-11-06
- AIFP warns: if Anthropic hits its 2x RSP acceleration threshold, superintelligence timelines may collapse — austinc3301 · 2026-11-06
- Grok and ChatGPT agree intelligence has limits; author predicts peak by 2030 — davidpattersonx · 2026-11-06
- World models can 'see into' any drawing, photo or render: a computer graphics renaissance — XRarchitect · 2026-11-06
- Eli Lifland: A 2x Progress Rate by Dec 2026 Means Roughly 4.5x Capability Uplift — inductionheads · 2026-11-06
- Rethinking predoc interviews in the AI era: agent transcripts and blindfolded video tests — aniketapanjwani · 2026-11-06