Open-source DKV framework cuts KV-cache memory for long-context local inference
Om_5000 · reddit · 2026-07-25
DKV (DifferentialKV) is an open-source framework for compressing KV cache in long-context local LLM inference.
The project focuses on reducing memory usage through anchor-based representations, joint low-rank compression, exact residual preservation, and sparse routed attention. It ships with a CLI, MLX backend, CUDA backend under validation, a technical report, and a fully open-source implementation. The author is looking for technical feedback and possible integrations with llama.cpp, vLLM, and SGLang.
Related event: Open-source DKV compresses KV cache for local long-context inference(2 posts)→
More from Infra
- Nearly half of Claude revenue now flows through cloud marketplace commitments, up from 11% — BenBajarin · 2026-10-26
- Anthropic ARR goes vertical after price cuts: tokens up 4x, $100B+ ARR back in play — GavinSBaker · 2026-10-26
- Alex Banks argues power caps GPU supply, real risk is demand cooling first — thealexbanks · 2026-10-26
- Analyst: Ant has the most fungible AI workload of any model provider today — BenBajarin · 2026-10-26
- Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26