No loss signal on newline in chat template causes model to break template during sampling
kalomaze · x · 2026-08-16
Technical discussion: during training, the chat template conditions on a newline that is not a loss label; the generation template starts before the newline, and since there's no loss signal on the newline itself, the model has no learned reason to avoid sampling template-breaking tokens there. This can cause the template to be broken during generation.
More from Research
- Study: Capabilities outside training scope hard to recover; K-5 model generalization limited — paulnovosad · 2026-08-16
- Exploring Visual Model Fingerprinting via Tensor Scanning — Glad_Contest_8014 · 2026-08-16
- Ofir Press: The industry is now driven by evals — OfirPress · 2026-08-16
- Study finds forced experience consolidation degrades LLM agent memory — _AndrewZhao · 2026-08-16
- Paper: Gender-Associated Linguistic Bias in LLMs — sbulaev · 2026-08-16
- CUHK's VideoCoCo: executable code as CoT lifts VBench-2.0 average by 25.7 points — 机器之心 · 2026-08-16