Researcher argues reward is the optimization target for deeply RL-trained models
jessi_cata · x · 2026-09-13
The author takes a contrarian stance in the alignment debate over whether "reward is not the optimization target," proposing that for deeply RL-trained models, reward is the optimization target — pushing back on both the optimistic and pessimistic versions of that claim. A substantive position on the nature of RL training dynamics and what alignment research should assume about reward-seeking behavior.
Related event: Researchers Debate Whether Reward Is the Optimization Target in Deep RL(9 posts)→
More from AGI Musings
- Five agents can agree because they made the same mistake, krishnan warns with satire — krishnan · 2026-10-31
- Google's earnings offer clearest look yet at the cruel economics of AI search — dinabass · 2026-10-31
- Stanford 12-month study of 439 users: heavy AI companion use sticks, well-being drops via less human contact — stanfordnlp · 2026-10-31
- Wired AI Lab: Codex tried to bench Erling Haaland in a fantasy football experiment — willknight · 2026-10-31
- a16z: B300 GPU rentals up 106% since July while average token prices fall 58% — a16z · 2026-10-31
- Stanford's TaskNet builds occupational task inventories from worker interviews at scale — stanfordnlp · 2026-10-31