Training method ranking for small models: skill distillation first, then SFT, then RL

ziv_ravid · x · 2026-11-19

A Hebrew X thread fragment answering "how exactly to train": skill distillation is considered the simplest and most convenient route, followed by SFT, and finally model distillation plus RL. You can layer on many other techniques as well — quantization, routing, pruning, caching, and more.

Related event: Practical Guide to Fine-Tuning Small Models(2 posts)→

Original post →

More from coding & agent

coding & agent channel →