Open-source DKV cuts KV-cache memory for local long-context LLM inference
Om_5000 · reddit · 2026-07-25
- DKV (DifferentialKV) is an open-source framework for KV-cache compression aimed at long-context local LLM inference.
- It uses anchor-based representations, joint low-rank compression, exact residual preservation, and sparse routed attention to cut memory usage.
- The project includes a CLI, MLX backend, a CUDA backend under validation, a technical report, and a fully open-source implementation.
- The author is looking for feedback on the architecture, benchmarking, and potential integrations with llama.cpp, vLLM, and SGLang.
Related event: Open-source DKV compresses KV cache for local long-context inference(2 posts)→
More from Infra
- Musk: robot RAM demand will dwarf everything else, and 100% of superintelligence compute goes to deep space — elonmusk · 2026-10-27
- RTX 5090 hits $7K as PC gaming community goes into 'denial' over AI boom — PAstynome · 2026-10-27
- Wan2GP lands on Pinokio: 1-click AI video on 6GB GPUs — cocktailpeanut · 2026-10-27
- Wan2GP AMD one-click installer lands on Pinokio, runs AI video on 6GB VRAM — cocktailpeanut · 2026-10-27
- AWS on agent prompt caching: explicit + implicit patterns to slash token costs — AWS ML Blog · 2026-10-27
- Thieves spoofing GPS steal $100M+ in data center cargo since June — Ars Technica AI · 2026-10-27