Context Language Models: training agents to natively manage their own context
omarsar0 · x · 2026-10-04
Elvis Saravia (DAIR.AI) highlights a rising trend of work — AutoHarness, AutoContext, AutoCompact — that trains models to natively take over what the harness traditionally does, with the newest work letting agents decide for themselves when to compact.
The companion paper is Context Language Models (CLMs) from the University of Washington, Meta Superintelligence Labs and MIT: models manage their own context by treating it as an editable file, replacing harness-defined compaction, offloading and retrieval rules.
Key results:
- Zero-shot: +11.4% accuracy with 21.5% fewer FLOPs on BrowseComp-Plus; +5% scores with 59% fewer FLOPs on 12-hour EdgeBench; 65% more improvement at equal compute on a 24-hour six-repo agent-swarm task.
- Learned strategies: skill-optimization-evolved instructions add up to 35.9 points on ContextBench held-out; online RL lifts Qwen3.5-9B by 47.6% on BrowseComp-Plus with 12% fewer FLOPs.
- Serving cost: mid-context edits break prefix caching; the proposed Suffix Cache Reuse cuts server-side compute 35% vs standard SGLang at matched performance.
- Risk: a model-editable context is a new channel for injected or self-written instructions to persist across turns.
Related event: AutoCompact Lets Agents Decide Context Compression Themselves(2 posts)→
More from coding & agent
- New TLAPS-Bench blog: formally proving correctness of real-world system specs with TLA+ — tianyin_xu · 2026-11-19
- Pierre releases Words, a streaming Markdown parser 2x faster than md4x and 20x faster than Streamdown — andersonbcdefg · 2026-11-19
- Ex-Claude superfan: getting the best from Astra/Sol requires relearning how you prompt — mariofilhoml · 2026-11-19
- Rivet's lesson: JEV doesn't reason—split severity classification into tiny yes/no prompts — yawnxyz · 2026-11-19
- 16x Microsoft MVP demos a native PowerShell agent built on Microsoft's Intelligent Terminal — dfinke · 2026-11-19
- Musk: Treat Grok Bot like a really smart employee, says most users underuse it — elonmusk · 2026-11-19