Projection sampling: transforming expert data so SFT learns new skills without forgetting

burkov · x · 2026-10-03

Frontier LLM training faces a persistent dilemma: teaching models new capabilities without degrading existing skills.

The article introduces and evaluates projection sampling, a pre-training data transformation framework: external expert demonstrations are transformed into trajectories closely matching the base model's internal distribution, allowing standard SFT to achieve strong generalization while preventing forgetting.

Related event: Pope says AI-generated art differs ontologically from human art, sparking debate(3 posts)→

Original post →

More from Research

Research channel →