Context Language Models: Letting LLMs Edit Their Own Context Boosts Accuracy 11.4% While Cutting FLOPs 21.5%

datawithsuman · x · 2026-10-05

A paper by Rulin Shao, Luke Zettlemoyer, Pang Wei Koh et al. (AI2/UW) introduces Context Language Models (CLMs): models that natively manage their own context by treating it as a file they can freely edit.

Key results:

Caveats: with a 128K budget the gap to summarization mostly closes (47.3 vs 47.8 on EdgeBench-10, though cost stays lower); some math results are single runs; editable context creates a new prompt-injection surface, as the authors flag.

Related event: Context Language Models: LLMs Edit Their Own Context, Gaining 11.4% Accuracy While Saving 21.5% Compute(7 posts)→

Original post →

More from coding & agent

coding & agent channel →