Before and After Training: Skill Distillation and Data Iteration

A thread outlines pre-training validation via skill distillation from trajectories rerun in context, and post-training iteration, noting that cleaning data usually beats tuning hyperparameters or switching models.

2026-11-19 ~ 2026-11-19 · 2 related posts