Practical Guide to Fine-Tuning Small Models
A thread advises starting with 30B models runnable on a single GPU for LoRA experiments, ranking skill distillation as the easiest training method, followed by SFT and distillation plus RL.
2026-11-19 ~ 2026-11-19 · 2 related posts
- Practical advice on fine-tuning small models: start around 30B, cheap LoRA experiments on one GPU — ziv_ravid · 2026-11-19
- Training method ranking for small models: skill distillation first, then SFT, then RL — ziv_ravid · 2026-11-19