Context Language Models: LLMs Edit Their Own Context, Gaining 11.4% Accuracy While Saving 21.5% Compute
Researchers from the University of Washington, Meta Superintelligence Labs, MIT and Trillium Labs (including Rulin Shao, Pang Wei Koh and Luke Zettlemoyer) published Context Language Models (CLM): the model's live context is treated as a file that the model itself decides to keep, delete or move to disk, replacing engineer-written summarization/pruning rules and fixed action menus. A zero-shot CLM outperforms SOTA context-management methods, gaining 11.4% accuracy on BrowseComp while saving about 21.5% compute.
Confirmed
- Paper is arXiv:2609.37725, authored by Rulin Shao, Pang Wei Koh, Luke Zettlemoyer et al., with UW, Meta Superintelligence Labs, MIT and Trillium Labs involved
- Key results: +11.4% accuracy on BrowseComp with 21.5% compute savings
- The method lets the model rewrite context like a file, absorbing the external harness; zero-shot CLMs beat SOTA context-management baselines across multiple tasks
Why it matters
- Conventional agents rely on hand-written context-management scripts; CLM hands this to the model itself. Observers such as danielmac8 argue the model "absorbing the harness" could reshape agent engineering
- Higher accuracy with less compute shows autonomous context management has practical payoff, not just conceptual novelty
- If the approach generalizes, much external management code in agent frameworks could be simplified or removed
2026-10-03 ~ 2026-10-05 · 7 related posts
Primary sources
- Context Language Models: Letting LLMs Rewrite Their Own Context Cuts FLOPs 21.5% While Boosting Accuracy 11.4% — RulinShao ·
- Meta and UW's Context Language Models: agents that edit their own context gain 11.4% accuracy on 21.5% less compute — alex_verem ·
- Context Language Models paper: LLMs natively manage their own context as editable files — alex_verem ·
- Context Language Models: letting an LLM edit its own context and absorb the harness — daniel_mac8 · 2026-10-03
- Context Language Models: letting an LLM edit its own context and absorb the harness — daniel_mac8 · 2026-10-03
- [source] Meta and UW's Context Language Models: agents that edit their own context gain 11.4% accuracy on 21.5% less compute — alex_verem · 2026-10-05
- [source] Context Language Models paper: LLMs natively manage their own context as editable files — alex_verem · 2026-10-05
3 near-duplicate retellings: RulinShao · datawithsuman · datawithsuman