Context Language Models paper: LLMs natively manage their own context as editable files

alex_verem · x · 2026-10-05

The arXiv paper (2609.37725) behind the CLM thread, by Rulin Shao, Pang Wei Koh, Luke Zettlemoyer et al.: models treat context as a file and make unrestricted updates, with multi-agent contexts coexisting as files.

Numbers from the abstract: 11.4% higher accuracy and 21.5% fewer FLOPs on BrowseComp-Plus; +5 points with 59% fewer FLOPs on 12-hour EdgeBench; 65% greater improvement on a 24-hour multi-repo agent-swarm task. Skill-optimized natural-language steering adds up to 35.9 held-out points; online RL lifts Qwen3.5-9B by 47.6% with 12% fewer FLOPs. Suffix Cache Reuse cuts server-side compute by 35% vs standard SGLang.

Related event: Context Language Models: LLMs Edit Their Own Context, Gaining 11.4% Accuracy While Saving 21.5% Compute(7 posts)→

Original post →

More from coding & agent

coding & agent channel →