Pretraining as Transformation, Post-training as Translation
A new framing suggests pretraining acts as a transformation from random initialization to a structured model of the data distribution, while post-training works like translation, reweighting the distribution along verifier preferences. This also explains certain RL phenomena as post-training inducing a meta-strategy over representations and policies.
2026-08-27 ~ 2026-08-27 · 2 related posts
- Opinion: Pretraining is transformation, post-training is translation — Liu_eroteme · 2026-08-27
- Pretraining Builds Representations, Posttraining Induces Metapolicy — Liu_eroteme · 2026-08-27