Context Language Models: Letting LLMs Rewrite Their Own Context Cuts FLOPs 21.5% While Boosting Accuracy 11.4%
RulinShao · x · 2026-10-05
Researchers from UW, Meta Superintelligence Labs, MIT and others introduce Context Language Models (CLMs): instead of relying on an external harness, the model treats its context as a file and directly rewrites it, learning when to compress, delete, preserve or reorganize information — naturally extending to multi-agent setups where agent contexts coexist as files.
Key results:
- Zero-shot, CLMs beat SOTA context-management strategies: +11.4% accuracy with 21.5% fewer FLOPs on BrowseComp-Plus; +7% scores with 59% fewer FLOPs on 12-hour EdgeBench; 65% greater improvement at matched compute on a 24-hour multi-repo agent-swarm task.
- Steering CLMs with natural-language instructions evolved via a standard skill-optimization loop improves held-out accuracy by up to 35.9 points.
- An online RL method lifts Qwen3.5-9B by 47.6% on BrowseComp-Plus with 12% fewer FLOPs.
- A co-designed Suffix Cache Reuse serving scheme cuts server-side compute by 35% vs. standard SGLang at matched performance.
Authors include Luke Zettlemoyer, Pang Wei Koh, and Nathan Lambert.
More from coding & agent
- New TLAPS-Bench blog: formally proving correctness of real-world system specs with TLA+ — tianyin_xu · 2026-11-19
- Pierre releases Words, a streaming Markdown parser 2x faster than md4x and 20x faster than Streamdown — andersonbcdefg · 2026-11-19
- Ex-Claude superfan: getting the best from Astra/Sol requires relearning how you prompt — mariofilhoml · 2026-11-19
- Rivet's lesson: JEV doesn't reason—split severity classification into tiny yes/no prompts — yawnxyz · 2026-11-19
- 16x Microsoft MVP demos a native PowerShell agent built on Microsoft's Intelligent Terminal — dfinke · 2026-11-19
- Musk: Treat Grok Bot like a really smart employee, says most users underuse it — elonmusk · 2026-11-19