CurvDelta Paper: Low-Curvature Update Space Preserves LLM Capabilities While Learning and Unlearning

Zeguan Xiao · hf · 2026-11-19

A paper from Shanghai University of Finance and Economics, Alibaba and SUSTech introduces CurvDelta and CurvLoRA, which bake capability preservation into the parameterization of LLM updates.

By analyzing the local geometry of the preservation constraint around the pre-adaptation model, the methods construct a low-curvature space where parameter updates leave the preservation loss nearly invariant — made practical via Kronecker-factored approximate curvature (K-FAC) and a factorized projection, with CurvLoRA as a low-rank variant in the same geometry.

On mathematical reasoning, code generation, and knowledge unlearning benchmarks, both achieve competitive downstream performance while better preserving general capabilities, offering a favorable acquisition-retention trade-off.

Original post →

More from Research

Research channel →