NVIDIA researchers show KV caches transfer between models via closed-form linear mapping
anselm · x · 2026-09-24
NVIDIA researchers found a way to transfer KV caches directly between models in the same family (e.g., Qwen3 14B → 32B), skipping the costly prefill when swapping from a small model to a larger one mid-conversation.
- Problem: Switching models mid-chat forces the larger model to recompute the entire history, killing latency and compute budgets.
- Discovery: KV caches across model sizes are linearly related—a single small-model layer predicts 56% of the variance in the big model's keys.
- Method: A closed-form linear mapping transfers the small model's "memory" into the larger model without re-reading the prompt.
- Implication: Could cut prefill costs and time-to-first-token for LLM serving, if it generalizes across model families.
More from Infra
- UK Sovereign AI invests in Fractile to build next-gen frontier AI chips — charlieharris01 · 2026-11-11
- What happened when I moved vector retrieval into Snowflake SQL instead of orchestrating RAG outside — dima806_dima · 2026-11-11
- AI chip startup Fractile raises $720M Series C for DRAM-on-processor design — Tom_Westgarth15 · 2026-11-11
- Mobile AI Factory: agentic pipeline builds on-device SLMs that beat 5-shot frontier models — kalyan_kpl · 2026-11-11
- Vivo's Starlink direct-to-cell service hits 10M users in two weeks across Brazil — XFreeze · 2026-11-11
- Nan Ya Plastics to spend NT$11.53B on fifth glass fabric plant on AI server demand — zephyr_z9 · 2026-11-11