NYU paper: LLMs can generate replay and pick targets to preserve their own skills

newyorkuniversity · hf · 2026-10-12

A New York University paper, "Self-Preservation in Language Models," studies forgetting in continual learning: tasks used for preserving capabilities cover only part of what a model knows, while forgetting can extend beyond them. The authors organize task descriptions and examples into Task Banks that guide replay generation by a frozen model, running continual instruction tuning with Qwen models of varying sizes.

Key findings: (1) the new task and learning rate shape which prior capabilities degrade, and tracking previously correct answers reveals losses that average accuracy conceals; (2) generated replay reduces forgetting on tasks outside the bank, and combining partial banks strengthens this protection beyond either alone; (3) the model can help choose replay targets—selection informed by the new task improves mean held-out retention over random selection at 4B with similar new-task gains, though smaller models show no advantage. Under a fixed example budget, swapping new-task examples for replay improves preservation at the cost of smaller new-task gains. Overall, models can support broader self-preservation by generating replay data and, in some settings, choosing what to replay.

Original post →

More from Research

• NYU's DRMoET boosts robust MoE training, lifting 10.3B model average to 0.6767 — newyorkuniversity · 2026-10-12
• Weight decay causally controls the life cycle of massive activations, paper shows — Aimpoint-Digital · 2026-10-12
• SAGE finds unsupervised salient factors in images, hitting 0.950 probe accuracy on FFHQ glasses — Shuang Liang · 2026-10-12
• NVIDIA's ELR removes the one-loop tax: Koi-4B scores 18 at one loop, 29 at four loops — nvidia · 2026-10-12
• U-Space: label-free, single-pass uncertainty estimation for reasoning LLMs beats multi-pass baselines — AdRevolutionary3242 · 2026-10-12
• DeCoPrune: training-free pruning cuts video diffusion memory tokens 85%, 4.14x faster — XingangP · 2026-10-12

Research channel →