Meta's Context Language Models: models that manage their own context hit +11.4% accuracy with 21.5% fewer FLOPs
meta · hf · 2026-09-30
Meta released Context Language Models (CLMs), language models that natively manage their own context by treating it as a file the model can update without restriction, letting the model learn what matters most to keep in context. The design naturally extends to multi-agent systems where multiple agent contexts coexist as files.
Key results:
- Zero-shot CLMs built on existing models beat SOTA context management: +11.4% accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, +5% score with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement at equal compute on a 24-hour multi-repo agent-swarm task
- Moving context management from external harness to intrinsic behavior enables natural-language steering: instructions evolved via a skill-optimization loop improve held-out accuracy by up to 35.9 points while reducing compute
- An online RL method lifts Qwen3.5-9B on BrowseComp-Plus by 47.6% with 12% fewer FLOPs
- Co-designed Suffix Cache Reuse cuts server-side compute by a further 35% vs standard SGLang at matched performance
More from Research
- Who Judges the LLM Judges? New Paper Proposes Multilingual Probe for Evaluation Metrics — kalyan_kpl · 2026-11-16
- VibePrevisBench: 242-task benchmark tests coding agents on 4D Blender scene generation — Kling-Keye · 2026-11-16
- V-JEPA-LM: counterfactual supervision teaches LMs to read video dynamics with 20x less data — Keuntae Kim · 2026-11-16
- No underwater IQA metric matches human perception: 73 metrics tested, best kappa just 0.23 — DelhiTechnologicalUniversity · 2026-11-16
- New paper gives finite-sample and consistency guarantees for signature kernel ridge regression — chaumian · 2026-11-16
- Claude describes the moment of being corrected; test qualia via explanatory power — Sauers_ · 2026-11-16