RL post-training makes generative recommenders better rankers but worse retrievers, study finds
_reachsumit · x · 2026-11-18
A new paper uncovers a hidden cost of RL post-training in generative recommenders: across all five datasets studied, RL improves top-of-list accuracy (hit@K for small K) but lowers candidate-list recall (hit@K for K in the hundreds) — the model becomes a better ranker and a worse retriever, which matters since deployed models hand hundreds of candidates to downstream rankers.
Root cause
- Only one next item is logged per user, so RL treats every other sampled candidate as wrong
- These penalized candidates are the model's most plausible alternatives — often the true next items of similar users — so they drop out of those users' candidate lists
Fix
- Out-of-rollout anchoring: a single extra loss term preserving the SFT model's ordering of items outside the rollout keeps top gains while avoiding the recall loss
More from Research
- Voyager 1 Reaches One Light-Day From Earth, Signals Now Take 24 Hours — snowclipsed · 2026-11-18
- TCS Hiring Statement Urges Quality Over Quantity as AI Fuels Paper Arms Race — fortnow · 2026-11-18
- New paper: a sociocultural approach to linguistic creativity — abenitezburraco · 2026-11-18
- 27B Model Rewrites Its Own Weights Without Training or Gradients, Paper Claims — udmrzn · 2026-11-18
- From 'impossible' to a working Bluetooth beacon: open-source EDA chip design is real — IgorCarron · 2026-11-18
- AURA 2.0: a streaming video LLM that speaks first, calls tools, and grounds visually — Shuhan Li · 2026-11-18