KVCache Meetup Korea dives into LLM serving: CXL pooling, roofline analysis, batch scheduling
PyTorch · x · 2026-11-03
PyTorch recaps the KVCache Meetup Korea, co-hosted by PyTorchKR and XCENA, with talks from XCENA, SK hynix, FuriosaAI, and LMCache on LLM serving architecture trade-offs.
Topics covered:
- CXL memory pooling with Maru
- Roofline analysis of GPU compute vs HBM bandwidth in prefill and decode
- Batch scheduling on RNGD
- Process-isolated cache serving with LMCache MP Mode
Key takeaway: real-world inference optimization requires collaboration across every tier of the stack, from accelerators and memory architectures to runtime engines.
More from Infra
- Olla v0.1.0: open-source LLM proxy unifies Ollama, vLLM, SGLang with failover — 2shanigans · 2026-11-03
- Gamers Get 'No RAM For You' as Memory Prices Spiral — Cyborgized · 2026-11-03
- Cache price beats input price for coding agents: one Sonnet miss costs as much as 25 hits — zitoliv · 2026-11-03
- Local Office brings fully local doc AI to Windows ARM64: Qwen3 4B at ~80 tok/s on NPU — HowDevelop · 2026-11-03
- Comparing RunPod, Vast and owned boxes for all-day Comfy video generation workloads — EducationalGroup7245 · 2026-11-03
- Report: DeepSeek V5 to be pretrained almost entirely on Huawei Ascend chips, shunning Nvidia — kimmonismus · 2026-11-03