Context Language Models: Meta-linked team lets LLMs manage their own context, cutting FLOPs 21% with higher accuracy

armand_ruiz · x · 2026-10-06

A 13-author paper led by Rulin Shao with Luke Zettlemoyer and Pang Wei Koh (Meta Superintelligence Labs among affiliations) introduces Context Language Models (CLMs): LLMs that natively manage their own context by treating it as a file the model can freely edit.

Key points:

Model learns what to keep in context; naturally extends to multi-agent setups where each agent's context is a file
Zero-shot CLMs beat SOTA context-management strategies: +11.4% accuracy with 21.5% fewer FLOPs on BrowseComp-Plus; +5% score with 59% fewer FLOPs on 12-hour EdgeBench; +65% improvement at equal compute on a 24-hour multi-repo agent-swarm task
Natural-language instructions evolved via a skill-optimization loop improve held-out accuracy by up to 35.9 points
Online RL lifts Qwen3.5-9B by 47.6% on BrowseComp-Plus with 12% fewer FLOPs
Suffix Cache Reuse serving co-design cuts server-side compute 35% vs standard SGLang

The shift moves context management from hand-engineered harnesses into intrinsic model behavior.

Original post →

Context Language Models: Meta-linked team lets LLMs manage their own context, cutting FLOPs 21% with higher accuracy

More from coding & agent