PyTorch Conference poster to break down KV cache offloading: when CPU beats shared storage
PyTorch · x · 2026-10-20
PyTorch announced that Bongwoo Bak of Moreh will present a poster, "Where the Blocks Live," at PyTorch Conference North America (San Jose, Oct 20-21). It covers how vLLM offloads KV cache to CPU or storage when GPU memory fills up, and how llmd tracks every cache block to route requests to their cache — showing when CPU offloading suffices and when shared storage pays off. Registration runs up to $999 ($249 academic).
More from Infra
- Diamond-embedded copper vias shown to boost chip thermal conductivity — jwt0625 · 2026-10-20
- Stealth startup Physis debuts world foundation model play with PhysisServe, core tech headed to vLLM-Omni — vllm_project · 2026-10-20
- Barcelona Supercomputing Center AI Factory Accelerator Names Three AI Champions — NVIDIA Blog · 2026-10-20
- Luma's MARBLE: exact sequence parallelism for packed multimodal masks, wins 47/48 vs FlashMask+Ulysses — mrjonfinger · 2026-10-20
- Dual R9700 layer-split side effect: per-card load under 50%, fans stay quiet — yahbluez · 2026-10-20
- What Gen Models Run Well on a 3060/4060? Reddit Weighs Local Setups — Haghiri75 · 2026-10-20