Training thread wrap-up: evals and data matter most, training is the easy part
ziv_ravid · x · 2026-11-19
Concluding a long thread on model training, the author's key takeaway: most of the effort should go into evals and data — the training itself is the simple part.
On how to train, the recommended ladder:
- Skills distillation: considered the simplest and most convenient approach
- Then SFT (supervised fine-tuning)
- Finally model distillation and RL
Extra techniques like quantization, rerouting, pruning and caching can be layered on. The closing advice: don't let the model fool you — invest in evals and data quality first.
Related event: Engineer's Long Thread Breaks Down Enterprise Post-Training(2 posts)→
More from Research
- Sandia Labs' VASt model predicts high-entropy alloy properties from tens of samples — CurieuxExplorer · 2026-11-19
- FoodTruck Bench, an LLM agent business-sim benchmark, lands on Papers with Code — NielsRogge · 2026-11-19
- MIT reads and steers LLMs' internal physics intuition via hidden-state shifts in Gemma — ProfBuehlerMIT · 2026-11-19
- harness-hub open-sources RL-trained LoRA adapters tuned per agent harness, recipe coming Friday — yb2698 · 2026-11-19
- Freenet releases new video on network routing with old-school ML — Sanity · 2026-11-19
- Empire AI creates 120+ early-career research positions in New York, Cornell deadline Dec 15 — dianarycai · 2026-11-19