KVMem: virtualizing million-token agent workspaces on a 24GB consumer GPU
rohanpaul_ai · x · 2026-09-23
The arXiv paper "KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU" introduces KV-context virtualization: overflowed agent workspace history is paged as KV state across GPU memory, host RAM, and NVMe, with lightweight model-native attention-space indexes selecting relevant blocks to form a query-dependent execution view bounded by the native context window.
- Preserves fine-grained execution evidence vs. summarization, and avoids repeated prefilling
- Beats compaction baselines on LongMemEval, MemoryAgentBench, and AgentLongBench at up to 1M-token histories
- DeepSWE with Qwen3.8-27B: task success up from 43.8% to 48.4%
- A 24GB RTX 5090 laptop runs Qwen3.6/3.8-27B NVFP4 with MTP, virtualizing a 1M-token workspace — 4× the native 256K window
Related event: KVMem Virtualizes Million-Token Agent Workspaces on Consumer GPUs(2 posts)→
More from coding & agent
- Dolphin creator launches With, a Rust-safe systems language with zero ceremony — QuixiAI · 2026-09-23
- With lang site: empty folder to native window in 28s, C libs without bindings — QuixiAI · 2026-09-23
- DIY visual Siri: wiring Ling 3.0 Flash VL into a China iPhone via Shortcuts and OpenRouter — alifcoder · 2026-09-23
- Agent tasks can spiral into surprise bills: user calls for spend caps and auto-stop — yi111 · 2026-09-23
- Security audit: autonomous research program XBOW credited with ~12 upstream bug fixes — moyix · 2026-09-23
- Telling agents to use formal verification helps them write better code — sh_reya · 2026-09-23