CMU NLP lecture: keeping only the teacher's top token can reverse a student's learning signal

CatAstro_Piyush · x · 2026-11-13

Notes from CMU Advanced NLP Lecture 8 (fine-tuning, instruction tuning, distillation) with an accompanying article: in a constructed example, soft-target cross-entropy gives logit gradients (−0.1, −0.05, +0.15), while a one-hot top-token target gives (−0.5, +0.25, +0.25) — token B flips sign, so keeping only the teacher's top token can reverse the student's learning signal. The same holds for complete answers: sampling from the unchanged teacher estimates teacher-to-student cross-entropy, while beam search, temperature changes and filtering shift the target distribution. The lecture also separates what can change from what is scored: LoRA restricts weight updates, response-loss masks select prediction positions, frozen weights still participate in computation, and prompts are excluded from loss. Code included.

Original post →

CMU NLP lecture: keeping only the teacher's top token can reverse a student's learning signal

More from coding & agent