KVMem: virtualizing million-token agent workspaces on a 24GB consumer GPU
rohanpaul_ai · x · 2026-09-23
The arXiv paper "KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU" introduces KV-context virtualization: overflowed agent workspace history is paged as KV state across GPU memory, host RAM, and NVMe, with lightweight model-native attention-space indexes selecting relevant blocks to form a query-dependent execution view bounded by the native context window.
- Preserves fine-grained execution evidence vs. summarization, and avoids repeated prefilling
- Beats compaction baselines on LongMemEval, MemoryAgentBench, and AgentLongBench at up to 1M-token histories
- DeepSWE with Qwen3.8-27B: task success up from 43.8% to 48.4%
- A 24GB RTX 5090 laptop runs Qwen3.6/3.8-27B NVFP4 with MTP, virtualizing a 1M-token workspace — 4× the native 256K window
Related event: KVMem Virtualizes Million-Token Agent Workspaces on Consumer GPUs(2 posts)→
More from coding & agent
- YC-backed Roma turns your to-do list into an agent command center across Gmail and Notion — testingcatalog · 2026-11-12
- Boubou: open-source CLI turns your laptop into a fully local voice AI hub — alexcovo_eth · 2026-11-12
- User automates entire video clipping workflow with Nous Research's Hermes agent — Teknium · 2026-11-12
- Tool-call failure rates jump 14x across providers, charts pinpoint inference stack updates — remilouf · 2026-11-12
- As models improve, writing good evals gets harder and more important — amankhan · 2026-11-12
- Prime agent ships with day-one multi-agent communication, runs in Minecraft unmodified — sarahcat21 · 2026-11-12