Meta's Context Language Models: models that manage their own context hit +11.4% accuracy with 21.5% fewer FLOPs
meta · hf · 2026-09-30
Meta released Context Language Models (CLMs), language models that natively manage their own context by treating it as a file the model can update without restriction, letting the model learn what matters most to keep in context. The design naturally extends to multi-agent systems where multiple agent contexts coexist as files.
Key results:
- Zero-shot CLMs built on existing models beat SOTA context management: +11.4% accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, +5% score with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement at equal compute on a 24-hour multi-repo agent-swarm task
- Moving context management from external harness to intrinsic behavior enables natural-language steering: instructions evolved via a skill-optimization loop improve held-out accuracy by up to 35.9 points while reducing compute
- An online RL method lifts Qwen3.5-9B on BrowseComp-Plus by 47.6% with 12% fewer FLOPs
- Co-designed Suffix Cache Reuse cuts server-side compute by a further 35% vs standard SGLang at matched performance
Related event: Meta Introduces Context Language Models (CLM), Open-Sourced(3 posts)→
More from Research
- Physics-aware losses keep grain boundaries real when AI generates alloy microstructures — bravo_abad · 2026-09-30
- SOSP26 Paper YoloFS Targets Agent Filesystem Misuse, Built From 290 Real Incident Reports — tianyin_xu · 2026-09-30
- Agents can delete their own logs: Claude Code, Codex, others fail trace integrity, paper finds — maksym_andr · 2026-09-30
- NUS Proposes StoryEngine: A State-Grounded Agentic Framework for Coherent Long-Form Video Storytelling — NationalUniversityofSingapore · 2026-09-30
- AnyStep-WAM Cuts Denoising Steps by Up to 85% in World Action Models Without Losing Success Rate — Rui Wang · 2026-09-30
- Sys1Cal-v1 Dataset Finds Jev Suppresses a Third Truth Value, Boosting Soft Accuracy to 0.978 — Riccardo Porcedda · 2026-09-30