natolambert: importance-sampling fixes for train-inference mismatch dominate RL research
natolambert · x · 2026-10-31
AI researcher Nathan Lambert notes that while reading the Score Centering RL paper, he found a great appendix table comparing importance-sampling-based methods for train-inference mismatch in RL — which he considers the biggest focus of RL algorithm research since the GRPO takeoff.
More from Research
- CMU paper unifies pretraining, SFT and RL into one likelihood-based training algorithm — rohanpaul_ai · 2026-10-31
- 48-page deep dive: why modern LLM RL is secretly off-policy and 14 ways to fix it — cwolferesearch · 2026-10-31
- Google Health: AI Tools Aim to Support Psychiatrists Where Conversation Is Diagnosis — SRSchmidgall · 2026-10-31
- Benchmarking 'scoping': five ways to confine LLMs to a bounded operational domain — burkov · 2026-10-31
- Meta paper: looping transformer weights adds depth without more parameters, up to 32 loops — iScienceLuvr · 2026-10-31
- Simple neg-log-odds loss swap improves every training attribution method tested — davidbau · 2026-10-31