Before and After Training: Skill Distillation and Data Iteration
A thread outlines pre-training validation via skill distillation from trajectories rerun in context, and post-training iteration, noting that cleaning data usually beats tuning hyperparameters or switching models.
2026-11-19 ~ 2026-11-19 · 2 related posts
- Test before you train: distill skills from traces into context and rerun your eval — ziv_ravid · 2026-11-19
- After training it's iteration: cleaning data usually beats tweaking hyperparameters — ziv_ravid · 2026-11-19