VFold paper halves LLM value cache with symmetry-aware cross-layer merging, keeps 98% quality

StellaLisy · x · 2026-11-05

Researchers from Johns Hopkins and George Mason released VFold: Symmetry-Aware Cross-Layer Value Cache Compression (arXiv:2610.12338). Key points:

Exploits cross-layer similarity in the value cache with a symmetry-aware merging strategy, cutting the value cache in half while retaining over 98% of full-cache performance;
Composes with existing cache compression techniques—high-ratio quantization or key cache pruning—to reach compression ratios neither achieves alone, at minimal extra cost;
Requires no architectural changes to the LLM, avoiding performance degradation and architectural overhead during decoding.

The authors argue the value cache contains a major source of underutilized capacity, offering a simple yet effective direction for scaling context windows under memory constraints.

Original post →

VFold paper halves LLM value cache with symmetry-aware cross-layer merging, keeps 98% quality

More from Infra