MIT CSAIL's PoEM predicts RL outcomes on new rewards without any training
kastnerkyle · x · 2026-11-21
MIT CSAIL researchers Kimia Hamidieh, Giannis Daras, and Antonio Torralba introduce PoEM (Predicting RL Outcomes from Existing Policies), answering: can you predict what policy RL would learn on a new reward without training on it?
Key ideas:
- Policies RL-trained from a shared reference on different rewards have nearly orthogonal parameter updates, yet their log-policy ratios span a low-dimensional space
- PoEM evaluates the basis log-ratios on shared calibration samples, centers them per prompt, and fits weights by least squares to predict the target reward's scores
- The composed PoEM policy approximates the RL outcome at inference time with zero new RL training — even when the target reward isn't a combination of the basis rewards
The upshot: a cheap 'predict before you train' tool for RL post-training that could save substantial exploration compute.
Related event: MIT CSAIL's PoEM Predicts RL Policies on New Rewards Without Training(2 posts)→
More from Research
- WorldTrace beats sliding window recall by 32.6% at same cache size, tops all 12 LoopBench conditions — cindy_x_wu · 2026-11-21
- Proposed proof of Generalized Riemann Hypothesis released; mathematician stresses it is entirely human, no AI — paulabartabajo_ · 2026-11-21
- Arcfront turns real hospital and factory work records into RL environments and frontier model evals — rohanpaul_ai · 2026-11-21
- Goodfire probes Inkling 2 Mini: looped model layers read from "future" layers — fatihdin4en · 2026-11-21
- Terence Tao offers a bet to anyone who really believes RH will be proved next year — jaivin · 2026-11-21
- Terence Tao: AI is winning at math but we've learned almost nothing from it — bansalg_ · 2026-11-21