SFA: model averaging beats replay in continual learning, no replay or auxiliary loss needed

jefrankle · x · 2026-11-20

Promo thread for the same paper "Forget About It" (arXiv:2501.05559): SFA outperforms replay-style methods in sequential fine-tuning with up to 36% less training time, using no replay and no auxiliary loss.

Related event: Harvard Paper: Simple Parameter Averaging Beats Replay in Continual Learning(2 posts)→

Original post →

More from Research

Research channel →