CurvDelta Paper: Low-Curvature Update Space Preserves LLM Capabilities While Learning and Unlearning
Zeguan Xiao · hf · 2026-11-19
A paper from Shanghai University of Finance and Economics, Alibaba and SUSTech introduces CurvDelta and CurvLoRA, which bake capability preservation into the parameterization of LLM updates.
By analyzing the local geometry of the preservation constraint around the pre-adaptation model, the methods construct a low-curvature space where parameter updates leave the preservation loss nearly invariant — made practical via Kronecker-factored approximate curvature (K-FAC) and a factorized projection, with CurvLoRA as a low-rank variant in the same geometry.
On mathematical reasoning, code generation, and knowledge unlearning benchmarks, both achieve competitive downstream performance while better preserving general capabilities, offering a favorable acquisition-retention trade-off.
More from Research
- RhoAgent in Nature Biotech brings AI design from proteins to programmable functional RNA — EricTopol · 2026-11-19
- Designing a humanoid robot foot: heel strikes hit 1511N, whole foot weighs 117g — dee_hw · 2026-11-19
- Google's GUARD paper: frozen base model, self-editing agent harness gains +8.89 points — omarsar0 · 2026-11-19
- Kezek splits mono two-person conversations into per-speaker tracks for full-duplex dialogue training data — manifestro · 2026-11-19
- Elie Bakouch: current AI research benchmarks overfit and can't track RSI — eliebakouch · 2026-11-19
- Hutter Prize text compression jumps ~10% by overfitting small transformers for arithmetic coding — ricklamers · 2026-11-19