In-context learning is becoming continual learning: frontier LLMs ingest novel domains without fine-tuning

Ryoiki-Tokuiten · reddit · 2026-11-21

A Reddit long-post argues that in-context learning in frontier LLMs is severely underestimated: feed a curated 150k-token context (a compressed representation of task rules and output distribution) and the model fully ingests a completely novel domain and works with it as if trained on it — no fine-tuning.

Two firsthand examples:

The author also tested a continual-learning loop: read curated context → work → produce output → update context → repeat. Updated contexts measurably improved in rule adherence and consistency, though models still occasionally make wrong assumptions. Are these glimpses of continual learning?

Original post →

More from AGI Musings

AGI Musings channel →