Mapping agent swarm KV cache in 3D over 24h: 94 to 848 out/s with KV offload
AccBalanced · x · 2026-11-16
Follow-up showing kvlab's 3D visualization of agent swarm KV cache usage across thousands of long-context turns over 24+ hours: GPU HBM-only hits 94 out/s, HBM+DRAM offload 729 out/s, and HBM+WEKA AMG 848 out/s, broken down by TTFT and input token source (WEKA/DRAM/HBM/prefill).
More from Infra
- WEKA open-sources kvlab: replay real agent traffic to benchmark KV cache offload — AccBalanced · 2026-11-16
- A shortlist of LLM inference throughput benchmarks: AIPerf, AgentX, llm-inference-bench — AccBalanced · 2026-11-16
- KubeCon's big theme is inference: a full-stack breakdown from DRA GPU slicing to vLLM and llm-d — AccBalanced · 2026-11-16
- Jensen Huang discusses his comeback as Nvidia CEO with allies in FT interview — SnoozeDoggyDog · 2026-11-16
- Is the GPU rental rate boom flattered by VC cash and spot pricing? — AccBalanced · 2026-11-16
- Is bad news for Jensen Huang also bad news for the AI industry? — TechCrunch AI · 2026-11-16