CMU paper unifies pretraining, SFT and RL into one likelihood-based training algorithm
rohanpaul_ai · x · 2026-10-31
A new Carnegie Mellon paper (Training Foundation Models with a Single Algorithm, arXiv:2610.40564) argues the staged separation of LLM training isn't fundamental.
Key idea
- By recasting a form of RL (MaxRL) as maximum-likelihood fitting to model-generated target distributions, pretraining, midtraining, SFT and RL all fit under one likelihood-based trainer ("unified training").
- Training becomes a data-mixing schedule — what data, in what proportions, when — with staged training as a special case.
Results
- On reasoning puzzles with a 124M model, the best blended runs beat the best staged runs and preserved general skills better.
- A 0.76B model that blended 5% midtraining data into math RL hit 32.69% on MBPP+ after coding RL, vs 29.81% for staged training.
The authors suggest treating your training pipeline as a data-mixing schedule rather than abrupt stage switches.
More from Infra
- New Yorker: The global chip supply chain is humanity's greatest collective achievement — anselm · 2026-10-31
- Intel B65 hits 62 tok/s on Qwen3.8-Flash-Next with XeStrata 4-bit offloading — SamSausages · 2026-10-31
- 16GB GPU runs Qwen3.8-27B at 110k context and 55 tok/s via iGPU display trick — jayecin · 2026-10-31
- Big Tech's future data center commitments top $3T, with Google alone over $1T — rohanpaul_ai · 2026-10-31
- Anthropic in talks to lease 920MW gas-powered data centre campus near Edmonton — rohanpaul_ai · 2026-10-31
- GPU price index tracks 114 rates from 25 vendors, free MCP server for agents — StasBekman · 2026-10-31