TRL v1.15 Enables Fused LM Head by Default, Cutting Peak VRAM up to 82%

Hugging Face's TRL v1.15 enables a fused LM head by default across SFT, DPO, KTO, GRPO, RLOO and distillation, cutting peak VRAM by up to 82% and extending trainable sequence lengths by roughly 7x.

2026-10-09 ~ 2026-10-09 · 2 related posts

• TRL v1.15 ships fused LM head: 82% less peak VRAM, 7x longer sequences — QGallouedec · 2026-10-09
• TRL v1.15 defaults to fused LM head, extending training sequences up to 6.9x — LysandreJik · 2026-10-09