KV cache transplants on Qwen3.8-27B: start at Q6, hand off to Q3, beat static quant
wadeAlexC · reddit · 2026-09-26
Inspired by the Cache-to-Cache paper (arXiv:2510.03215), which fuses KV caches between models via a trained converter, the author tested whether KV caches transfer across quantizations of the same model with no converter. On Qwen3.8-27B with NIAH-style tasks, dynamic strategies (start Q6K, swap to Q4KXL, then IQ3S, optionally quantizing KV cache to q80) fit in 24GB and beat running low-precision quants from the start on his benchmarks.
More from Infra
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Nvidia's supply commitments jump from $119B to $279B in one quarter, locking up 2027-28 memory — tengyanAI · 2026-11-11
- Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight — oguzeldereli · 2026-11-11