DeepMind researcher: add RL rollouts to pretraining and nothing fundamentally changes
burny_tech · x · 2026-09-27
DeepMind researcher Tomasz Korbak debates Melanie Mitchell on whether RL post-training moves LLMs beyond language modeling. His take: it's still modeling language from a slightly different distribution — the architecture and objective (for a fixed dataset) are basically the same, and adding RL rollouts to pretraining data wouldn't change the effect. He also argues the stochastic parrots paper made claims about the whole next-token paradigm without carving out this distinction.
More from Models
- Fable 5.2 Leads MazeBench Token Efficiency; Billion-Token Runs Underway — patience_cave · 2026-11-16
- Fable 5.2 Tops MazeBench With 21% After 90 Hours and $6,000 in Credits — patience_cave · 2026-11-16
- Tencent Hy4 flash and InternLM Sage-S2.1-Flash both released, tool-calling tests requested — MajesticPuzzler · 2026-11-16
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16