Researchers Debate the Boundary Between In-Context Learning and Continual Learning

This is a technical discussion among researchers, centered on the nature of in-context learning (ICL) and its relationship to continual learning. Participants: kalomaze, Beren Millidge, and teortaxesTex.

Confirmed

• In a discussion with Beren Millidge, kalomaze argued that current ICL is not a deliberately designed capability but a passive byproduct of next-token prediction (NTP) when the backbone gets more conditioning information; he said he hasn't seen anyone deliberately distill a classifier into multiple independent SGD/Adam training steps
• kalomaze further offered a technical conjecture: a sufficiently deep sequence model might, in the sense of log-probability shifts, emulate the behavior of certain restricted training families (such as attention-only rank-1 LoRA) on serialized samples, but doing so would require deliberate KL distillation
• Commenting on new attention methods, teortaxesTex asked: now that append-only inference is possible over extremely long agentic sequences, what is the essential difference from the repeatedly promised "continual learning"? He argued that even with retrieval, residual-stream capacity limits remain a constraint
• Beren Millidge responded that he doesn't believe such methods solve continual learning: attention-based ICL essentially just adds one vector per token into the residual stream, so its expressivity is ultimately limited—despite being extremely powerful already

Why it matters

• The discussion touches a key question in current long-context and agent research: there is a fundamental gap between the "in-context-style adaptation" enabled by ultra-long-context inference and genuine continual learning via weight updates
• Millidge's point about the expressivity ceiling of the residual stream offers a sober technical perspective on the popular expectation that long context can substitute for continual learning; kalomaze's conjecture about deliberately distilling training steps, meanwhile, suggests a possible research path toward bridging ICL and real gradient updates

2026-11-03 ~ 2026-11-03 · 6 related posts

• [source] Beren Millidge: attention-based ICL can't solve continual learning due to residual stream limits — BerenMillidge · 2026-11-03
• [source] teortaxesTex asks if append-only reasoning over long agentic sequences is just continual learning — teortaxesTex · 2026-11-03
• [source] kalomaze: In-context learning is an incidental byproduct of next-token prediction, not deliberate design — kalomaze · 2026-11-03
• kalomaze conjectures deep sequence models could mimic restricted training families like rank-1 attention LoRAs — kalomaze · 2026-11-03
• kalomaze conjectures ICL may mimic training steps via deliberate KL distillation — kalomaze · 2026-11-03

1 near-duplicate retellings: BerenMillidge