CacheGraft grafts a 4B model's KV cache into a 0.6B model to boost small-model RAG

_reachsumit · x · 2026-11-06

CacheGraft extends the Cache-to-Cache fusion framework to RAG: offline, each corpus passage is encoded into a fused KV cache between a 4B sharer and 0.6B receiver model, injected at query time with no large model needed. Across five QA benchmarks and two local judges it beats the text-only receiver baseline in 29 of 30 comparisons (17 statistically significant), zero-shot with respect to the fuser — promising for compute-constrained RAG.

Original post →

More from Research

• UWaterloo 3D prints Invar36 flexures with directly mounted interferometer optics — jwt0625 · 2026-11-06
• AMD paper: a single generalized MLP upscales textures on-the-fly at render time, saving VRAM and disk — ssh4net · 2026-11-06
• OpenAI researcher invokes 2019 double descent paper in scaling-law spat — willdepue · 2026-11-06
• Encord debuts REACH egocentric hand-tracking model, cutting errors up to 70% — Scobleizer · 2026-11-06
• Only 16% of AI benchmarks measure organizational difficulties of real jobs — ben_burtenshaw · 2026-11-06
• Skillware framework claims 97.3% cost cut as 3B-8B local models beat frontier LLMs — RossPeili · 2026-11-06

Research channel →