Samsung Pools KV Cache Using CXL
lauriewired · x · 2026-07-12
Samsung has published a paper/proposal detailing the offloading of KV-cache to a CXL memory pool.
- The experiment utilized an early CXL 2.0 ecosystem, older switches, and a relatively basic setup.
- By applying simple interleaving of the KV Cache across multiple CXL modules, the GPU supply remained stable, performing almost "like real DRAM".
- While the author notes this isn't SOTA, the implementation is highly reproducible, demonstrating that CXL already offers practical value for scaling inference memory.
More from Infra
- Sail equivalence proof and riscv-coq cross-checks uncover real RISC-V spec bugs — xennygrimmato_ · 2026-10-27
- xAI's Colossus 1 hits ~14.6 exaflops FP64, ~5x Top500 leader — cis_female · 2026-10-27
- Lambda runs 19 B200 nodes in a 16-node power budget, boosting throughput 24% — TheZachMueller · 2026-10-27
- Podcast: Micron's HBM roadmap, Meta's AI infra, and NVIDIA's supply chain lock-in — BenBajarin · 2026-10-27
- SGLang MegaKernel + PD Pushes the TCO-Latency Frontier on GB300, SemiAnalysis Benchmarks Show — AccBalanced · 2026-10-27
- Scaling self-hosted Convex: compiling TypeScript Convex functions to WASM — leventov · 2026-10-27