KVMem pages agent context overflow to KV state, beating compaction on DeepSWE
omarsar0 · x · 2026-09-09
Elvis Saravia highlights a new paper, KVMem, targeting inference efficiency for long-running agents whose workspaces outgrow the context window long before the task finishes.
Existing approaches fall short: compaction loses fine-grained execution evidence, while text retrieval re-prefills content the model already processed.
KVMem's approach:
- Keeps overflow as paged KV state, spread across GPU memory, host memory, and NVMe instead of discarding or converting to text
- Uses lightweight attention-space indexes native to the model to pick relevant historical blocks and materialize a query-dependent view that fits inside the native context window
Results: on the DeepSWE long-context test with Qwen3.8-27B, task success rises from 43.8% under compaction to 48.4%.
Local deployment also stands out — it runs Qwen3.6/3.8-27B NVFP4 with MTP on a laptop with a 24GB RTX 5090 while virtualizing a much longer context.
More from coding & agent
- Stripe built its 'company brain' Kai on open-source deepagents in 2 weeks — Hacubu · 2026-09-09
- Anthropic engineer's 14-minute video on Building Effective Agents is worth an hour — HeyAmit_ · 2026-09-09
- Agent Harness: The New Runtime for Agentic Engineering — Pavan_Belagatti · 2026-09-09
- Fuzzing their Rust Postgres rewrite surfaced 20+ bugs in Postgres itself — DanielLockyer · 2026-09-09
- Switching AI from GUI to CLI mysteriously tanked this Mac's battery life — DanielLockyer · 2026-09-09
- Microsoft hosts MCP Live! 4-hour livestream with GitHub, AWS, Okta and Anthropic speakers — lee_stott · 2026-09-09