CMU paper: one algorithm for pretraining and post-training — unified training beats staged pipelines
heghbalz · x · 2026-10-30
A CMU/Johns Hopkins/Aurora paper argues the staged pretraining→SFT→RL pipeline isn't fundamental. By deriving RL objectives as maximum-likelihood fitting to model-generated targets, all stages collapse into a single likelihood-based objective, unified training. Design choices reduce to which data to train on, in what proportion, and when. Controlled and real-world experiments show unified training improves the accuracy–compute tradeoff and task coverage at equal total FLOPs, while staged training becomes a special case of data mixing.
More from Models
- The meme is real: DeepSeek-led open-weight models hit 21 ops in Q3, beating all prior quarters combined — Jsevillamol · 2026-10-30
- Polymarket puts Google at 59% to lead the AI race by year-end — Polymarket · 2026-10-30
- Nanbeige4.3-33B with 512K context tops agent benchmarks, real-world feedback sought — xdcfret1 · 2026-10-30
- LLMs struggle badly on tasks with more than one right answer — andpoul · 2026-10-30
- Nace AI open-sources Drex-Max v1: 27B multimodal decision model scoring 255 options in one pass — techNmak · 2026-10-30
- 0.87MB Hindi typing model Lekhani beats a 14x larger model in-browser — SuB8u · 2026-10-30