Two papers show how pretrained models bypass the proven infeasibility of optimal class-incremental learning
khademinori · x · 2026-09-25
Framing this with his PhD thesis proving the infeasibility of optimal class-incremental learning, the author highlights two new papers explaining why pretrained models get past that barrier.
- The first shows fine-tuning erodes the generalization margin at a rate that grows with the learning rate.
- The second characterizes in closed form the per-stage offset that L2 regularization induces in sequentially trained cross-entropy heads, and introduces gauge anchoring, a constraint that removes it at negligible cost.
- Both results were checked against exact solves under a pre-registered protocol.
More from Research
- Continuous diffusion LM RePlaid accepted at NeurIPS 2026, matches discrete diffusion scaling — ArashVahdat · 2026-09-25
- AAAI 2027 phase 1 reviews are out — CSProfKGD · 2026-09-25
- NeurIPS AC Posts Meta-Review Claiming No Rebuttal Submitted—Before the Rebuttal Period Even Started — BlackHC · 2026-09-25
- MUX latent reasoning method wins NeurIPS Spotlight, cuts CoT length 3-6x — mmbronstein · 2026-09-25
- Stanford's 37K AI agents analyzed 57K clinical trials as a 'virtual biotech' — james_y_zou · 2026-09-25
- CVP from UC San Diego and Lambda beats Video-3D-LLM on all five 3D spatial reasoning benchmarks — TheZachMueller · 2026-09-25