We need a better taxonomy for "continual learning": five mechanisms, five tradeoffs

Typical-Scene-5794 · reddit · 2026-09-03

The author argues "continual learning" names a problem setting—acquiring knowledge after deployment without full retraining—not a mechanism, and proposes a five-way taxonomy:

1. Long-context ICL: frozen weights, learning from demos/feedback/history in context. Limit: huge working memory but no transfer to long-term memory.

2. Text optimization: rewriting the mutable text layer (system prompts, skill files, playbooks, memory stores, retrieval indices, harness code). Forgetting doesn't vanish—it moves to memory construction/retrieval.

3. Recurrent/architectural latent memory: task info written into evolving internal state with frozen weights (e.g., BDH-CQ-style self-modifying architectures); adaptation at inference via state updates alone.

4. Per-task test-time training: demos become a small training set, gradients applied then possibly discarded. ARC pipelines like HRM/TRM fit here; whether it's truly "continual" is doubtful without cross-task persistence.

5. Online parametric continual fine-tuning: persistent post-deployment gradients; challenges include test-time label/reward quality and avoiding capability loss (replay, regularization, isolation, sparse updates). Hybrids self-generate fine-tuning data in text but store updates in weights. Risks: catastrophic forgetting and updates not surviving base-model upgrades.

He invites disagreement and additions.

Original post →

We need a better taxonomy for "continual learning": five mechanisms, five tradeoffs

More from AGI Musings