First formal theory of how mid-training shapes RL post-training in LLMs
burkov · x · 2026-10-13
- LLMs increasingly rely on RL for code generation and multi-step reasoning, which typically requires a mid-training phase (continued pre-training on expert demonstrations) before RL post-training.
- Standard mid-training uses next-token prediction, treating every token as a separate decision — creating a huge decision space and long horizons, leaving its effect on downstream RL poorly understood and computationally costly.
- The article presents the first formal theoretical framework explaining how mid-training influences post-training RL, introducing "Reasoning as Action Abstraction" to ground the mechanism.
More from Research
- François Fleuret on why matrix multiplication keeps GPUs perfectly busy — francoisfleuret · 2026-10-13
- Specula has found 500+ bugs in distributed systems; Apache Ozone adopts it — tianyin_xu · 2026-10-13
- Researchers debate whether an n(log n)^a integer-multiplication result was misread — rrwilliams · 2026-10-13
- Reinventing the wheel: hand-rolled upscaling for tiny retro-console pixel sprites — ssh4net · 2026-10-13
- Preferred Networks' T3lescope reconstructs high-fidelity 3D scenes from multi-view images without per-scene optimization — pfnet · 2026-10-13
- ARCS benchmark: ambiguous text-to-SQL stumps models, best at just 51% accuracy — megagonlabs · 2026-10-13