Training method ranking for small models: skill distillation first, then SFT, then RL
ziv_ravid · x · 2026-11-19
A Hebrew X thread fragment answering "how exactly to train": skill distillation is considered the simplest and most convenient route, followed by SFT, and finally model distillation plus RL. You can layer on many other techniques as well — quantization, routing, pruning, caching, and more.
Related event: Practical Guide to Fine-Tuning Small Models(2 posts)→
More from coding & agent
- Hermes adapter solves a SWE-Bench Pro task in 35 tool calls where the base Qwen3.8 model failed — yb2698 · 2026-11-19
- NousResearch's Hermes adapter lifts Qwen3.8-27B agent scores: SWE-Bench Pro 29.4%→43.1% at 4× fewer tokens — yb2698 · 2026-11-19
- One Amazon EBS gp3 command scales a 10M-vector Weaviate index 26×, from 9.8 to 260 QPS — philipvollet · 2026-11-19
- Human Principal launches private beta: cryptographic proof a human is behind your AI agent — yenkel · 2026-11-19
- GPT-6.2 builds code-only 3D fairy town and a deep-sea civilization in 10–20 minutes — OpenAIDevs · 2026-11-19
- Designers are being asked to do interior design while ignoring the architecture — round · 2026-11-19