CMU paper unifies pretraining, SFT and RL into a single likelihood-based algorithm
CarnegieMellonU · hf · 2026-10-31
- Researchers from CMU, Johns Hopkins, and Aurora Innovation propose unified training: a single likelihood-based algorithm covering pretraining, midtraining, SFT, and RL.
- Key idea: derive RL objectives as maximum-likelihood fitting to model-generated target distributions, removing the algorithmic split between stages.
- Foundation-model training becomes a data-selection problem (what data, what proportions, when), with staged training as a special case.
- They instantiate a system mixing static data with RL generations, reporting improved efficiency and capability retention.
More from Research
- Fixing misaligned NVS benchmarks with image matching puts 3D Gaussians back on top — kwangmoo_yi · 2026-10-31
- NVS benchmarks are misaligned: proper pose alignment puts 3D Gaussians back at SOTA — kwangmoo_yi · 2026-10-31
- arXiv moderator: submission surge causes months-long holds, new moderators wanted — tdietterich · 2026-10-31
- arXiv policy checks reject class projects, proposals, surveys and position papers — tdietterich · 2026-10-31
- arXiv moderator explains the four main reasons papers get put on hold — tdietterich · 2026-10-31
- New preprint: closed-loop ephys successfully trains in vitro neuron cultures to change behavior — DoctorJosh · 2026-10-31