Context Language Models: LLMs that manage their own context beat SOTA with 21.5% fewer FLOPs
rohanpaul_ai · x · 2026-10-01
Context Language Models (CLMs)
Researchers including Luke Zettlemoyer and Pang Wei Koh propose language models that natively manage their own context by treating it as a file the model can freely edit—letting it learn what to keep, and extending naturally to multi-agent systems where agent contexts coexist as files.
Zero-shot CLMs beat SOTA context management:
- BrowseComp-Plus: +11.4% accuracy with 21.5% fewer FLOPs
- 12-hour EdgeBench: +5% score with 59% fewer FLOPs
- 24-hour multi-repo agent-swarm task: 65% greater improvement at equal compute
Trainable and optimizable:
- Steering via natural-language instructions evolved with a standard skill-optimization loop yields up to +35.9 points on held-out context-management tasks
- Online RL lifts Qwen3.5-9B on BrowseComp-Plus by 47.6% with 12% fewer FLOPs
- A Suffix Cache Reuse serving design cuts server-side compute 35% vs standard SGLang
More from coding & agent
- Dev argues Rust would be better if modules, not crates, were the compilation unit — zetalyrae · 2026-10-25
- Dev open-sources BizAgent framework where one AI lead agent manages a dozen AI employees — Low-Start2333 · 2026-10-25
- Formal Verification Yields to Static Analysis, Which Yields to Better Language Design — xennygrimmato_ · 2026-10-25
- Scanning 115,798 agent ledger rows: zero capability revocations, none with expiry — Affectionate-Sun265 · 2026-10-25
- 'Slurm's defense remains annoyingly strong': dev watches agents hit HPC wall — sloppenheimer · 2026-10-25
- 5 open-source tools turn user interviews into a full content marketing workflow — sven_ai · 2026-10-25