New theory paper argues SFT should reward any good answer, not imitate demonstrator distribution
burkov · x · 2026-10-20
- Andriy Burkov highlights a theory article on the fundamentals of supervised fine-tuning (SFT): most questions admit many equally valid answers, yet standard SFT minimizes log-loss, effectively cloning the human demonstrator's exact style and answer distribution.
- The authors argue this is the wrong target—success should mean producing any single good answer. Distribution-matching is unnecessarily strict for math, coding, and essays, and theoretically untenable under natural modeling assumptions.
- The article formalizes fine-tuning as imitation learning in contextual bandits to back the claim.
More from Research
- Roboticist slams ICLR 'agentic robot planning' papers for ignoring classical planning and scaling lessons — KyleMorgenstein · 2026-10-20
- Berkeley Team Teaching AI Language Like a Baby Wins Schmidt Sciences Funding — begusgasper · 2026-10-20
- Brain-behavior AI framework finds 3 transdiagnostic biotypes in youth with ADHD and anxiety, in Nature Mental Health — bttyeo · 2026-10-20
- HoliWorld: scalable auto-labeling pipeline turns egocentric video into 3D/4D data — liuziwei7 · 2026-10-20
- mjfit peg benchmark under fire: cylinder-peg setup embeds a strong prior — YouJiacheng · 2026-10-20
- Industrial overcapacity enables scientific discovery: five cases from Michelson to Meta datacenters — charlesxjyang · 2026-10-20