NVIDIA researchers show KV caches transfer between models via closed-form linear mapping
anselm · x · 2026-09-24
NVIDIA researchers found a way to transfer KV caches directly between models in the same family (e.g., Qwen3 14B → 32B), skipping the costly prefill when swapping from a small model to a larger one mid-conversation.
- Problem: Switching models mid-chat forces the larger model to recompute the entire history, killing latency and compute budgets.
- Discovery: KV caches across model sizes are linearly related—a single small-model layer predicts 56% of the variance in the big model's keys.
- Method: A closed-form linear mapping transfers the small model's "memory" into the larger model without re-reading the prompt.
- Implication: Could cut prefill costs and time-to-first-token for LLM serving, if it generalizes across model families.
More from Infra
- Industry responds to hyperscale RDMA paper with Multipath Reliable Connection on path to Ultra Ethernet — thoefler · 2026-09-24
- Amazon nearly doubles US data center capacity since ChatGPT launch, 69% ahead of Microsoft — anselm · 2026-09-24
- JVM runs NVIDIA Parakeet ASR at up to 2X the C++ reference speed, no GPU or Python needed — mkheck · 2026-09-24
- Run a Claude Code-like coding workflow for free locally with Ollama — Aiden_Tech_Ai · 2026-09-24
- _xjdr: TPUs performing well wasn't a given — time to look at v8s — _xjdr · 2026-09-24
- CoreML runs FP8-weight model on M6 chip in early patch test — AIFlow_ML · 2026-09-24