PoEM: predicting RL policy outcomes by composing models trained on other rewards
gowthami_s · x · 2026-11-21
A new paper introduces PoEM, a method that predicts the policy RL would learn for a new reward without actually running RL training. Key finding: even when reward functions aren't linearly connected, RL tends to produce log-policies that are approximately linearly related, so a new RL run can be approximated by composing models previously trained on a broad family of other rewards. The authors validate PoEM on both LLMs and diffusion models, with programmatic and neural rewards; qualitative comparisons show PoEM outputs closely matching real RL targets on image rewards like red fox, bald eagle, and lion.
Related event: MIT CSAIL's PoEM Predicts RL Policies on New Rewards Without Training(2 posts)→
More from Research
- NEJM Analysis of $45B in Medicare Payments: Pathology 99% AI-Exposed, Dermatology 7% — HealthcareAIGuy · 2026-11-21
- DeepMind spins up a math horizon repo amid unverified claims of Gemini solving open math problems — kimmonismus · 2026-11-21
- Why AI hasn't yet proven useful deep learning theory: toy models are too hard to design — jasondeanlee · 2026-11-21
- Cohere researcher talks rethinking coding-agent evaluation at Agent Eval Symposium — nouhadziri · 2026-11-21
- NVIDIA lays out 8 design guidelines for sparse attention, reusing selections for 2.4x faster Top-k — NVIDIAAI · 2026-11-21
- Tracking board logs Alpöge's proposed RH/GRH proof built with Claude, still unverified — latticecut · 2026-11-21