NWPU's ROSE framework (NeurIPS Spotlight) curbs forgetting in lifelong multimodal knowledge editing

机器之心 · wechat · 2026-11-21

A team at Northwestern Polytechnical University (advised by Profs. Song Lingyun and Shang Xuequn) presents ROSE, a lifelong knowledge-editing framework for multimodal LLMs accepted as a NeurIPS 2026 Spotlight, targeting catastrophic forgetting (new edits overwriting old ones) and locality erosion (updates damaging unrelated knowledge and pretrained abilities).

ROSE freezes the backbone and writes new facts into an auxiliary branch on the FFN down-projection: gradients are projected onto the orthogonal complement of historical edit subspaces to avoid conflicts, plus a dynamic risk mask limits which parameters update. At inference, visual/textual semantic prototypes gate whether an input touches edited knowledge, fusing branch features only where differences are significant.

After 1000 sequential edits on LLaVA-v1.5-7B in VLKEB, ROSE keeps 95.24% reliability (FT-L drops to 68.14%), beats LiveEdit on textual (92.74%) and multimodal (84.53%) generalization, and reaches 100% textual / 88.39% multimodal locality; single edits take 1.31s on one RTX 4090. Paper and code are open-sourced.

Original post →

More from Research

Research channel →