Draft-KV: Learning Useful Latent Communication Between Language Models
Linquan Wu, Shichang Meng, Tianxiang Jiang, Haoyu Yang, Peng Zhong, Fengming Zhu, Xi Peng, Linqi Song, Jacky Keung, Jingyu Zhang
cs.CL
2026-09-28
Draft-KV sends draft-time KV via a 1.05M interface. Frozen Qwen2.5-0.5B-Instruct hits 78.04% MMLU-Redux with Qwen3-8B, vs 37.45% alone and 36.40% with swapped messages.
If two language models collaborate in text, one has to decode its computation into tokens and the other has to encode those tokens again. Latent communication skips that round trip and ships hidden states or KV caches instead. Cache-to-Cache (C2C) projects a sharer's prompt cache into the receiver; LatentMAS relays shared KV without training. Both report higher receiver accuracy.
Higher accuracy does not mean the receiver used the message. Across five method-dataset pairs, replacing each message with one from an unrelated question moves accuracy by at most 0.60 points, while the same systems add as much as 15.44 points over the receiver alone. For C2C with Qwen2.5-0.5B-Instruct sharing to Qwen3-0.6B, MMLU-Redux gains 12.29 points as a system and -0.47 points from pairing; OpenBookQA is 13.40 / 0.00 and ARC-Challenge is 15.44 / 0.42. LatentMAS keeps 0.60 points of pairing gain out of 4.86 points of system gain on ARC-Challenge. Averaging the frozen interface over mismatched messages still retains 93% to 96% of C2C's Matched accuracy. Scaling the sharer from 0.6B to 8B raises standalone accuracy by 29.85 points and system accuracy by 3.54.
The interface can carry the gain while the sharer is interchangeable. The paper calls this the information-use gap and treats it as the design target: the message has to contain computation the sharer just ran on this question, and the receiver's answers have to move when that pairing is broken.
Draft-KV sends the key-value states formed at draft tokens while the sharer tries to answer, not the static prompt-position cache. Under causal attention, prompt KV never sees the answer that follows; draft KV has already mixed the question with the tokens written so far.
Both backbones stay frozen. Against Qwen2.5-0.5B-Instruct, the interface is 1,048,584 parameters, 348× smaller than C2C and 36× smaller than DLC. Size follows cache widths and the four layers, not model scale or message length. Tokenizers, depths, and KV-head counts may differ.
Training is staged because reading a message and using it to answer are different skills.
At inference the sharer drafts, the interface projects, and the receiver decodes with the packet held fixed.
Headline pair: Qwen3-8B sharer, frozen Qwen2.5-0.5B-Instruct receiver, same question on both sides.
| Method | MMLU-Redux | ARC-Challenge |
| Receiver-only | 37.45% | 40.10% |
| C2C Matched / Deranged | 36.97 / 36.45 | 40.53 / 39.42 |
| Same draft as text (T2T) | 72.05% | 84.73% |
| Sharer-only | 75.04% | 92.83% |
| Draft-KV Matched / Deranged | 78.04 / 36.40 | 93.77 / 38.57 |
Draft-KV beats the receiver in all 35 Public settings, ranks first in 30, and beats T2T in 33 by 5.85 points on average. Mean system gain is 24.24 points; mean pairing gain is 25.65; both are positive in every cell. C2C's mean pairing gain is 1.31 points, and its system gain is negative in 29 of 35.
With the interface size held fixed, scaling the Qwen3 sharer from 0.6B to 8B lifts MMLU-Redux from 46.11% to 78.04%. Mean system gain across five benchmarks rises 32.6 points, against 30.2 for T2T and 1.4 for C2C, and beats the sharer's standalone edge in 19 of 20 cells.
When gold evidence is split at random (HotpotQA, 2WikiMultihopQA), Draft-KV beats T2T in all 14 settings by 6.41 points and ranks first in seven, those seven also beating both models alone. Qwen3-8B→0.5B HotpotQA exact match is 42.48% against 32.28% receiver-only and 35.21% sharer-only. A deranged packet falls 10.98 points below Receiver-only here, because the packet is the receiver's only path to the sharer's evidence.
Swap draft-position KV for prompt-position KV on Qwen3-1.7B→0.5B and ARC-Challenge drops from 83.53% to 49.83%, pairing gain from 45.90 points to 0.17. The information-use gap reappears inside the same architecture. Dropping reconstruction or answer alignment hurts held-out MMLU-Redux more than the ARC split Stage 3 trained on. On ARC-Challenge the receiver recovers 24.1% of questions the sharer gets wrong and loses 1.1% of those it gets right.
Pairing gain is the check that content actually crossed. A large fraction of reported latent-communication lifts can come from interface adaptation, not from what the sharer computed.
Draft-KV meets that check with a frozen-backbone interface of about a million parameters, including across families. The fit is a small model that needs a larger model's just-run computation, or two models that hold different evidence. The bill is the draft: on Qwen3-8B→0.5B, Draft-KV does 2.02× the arithmetic of Sharer-only and 1.01× the latency, because 11.87 of 12.02 seconds are 368 memory-bound decode steps on the 8B. The interface itself is 0.88 GFLOPs and 0.63 ms; C2C's interface at the same slot is 118.3 GFLOPs and 35.6 ms. C2C skips drafting and runs at 0.009× Sharer-only latency. What shrinks is the adapter, not the draft. Content-dependent gains still buy a sharer decode.
There is no Limitations section. The setup supplies the constraints. Each model pair gets its own interface. Stage 3 supervises only ARC train and validation; transfer to MMLU-Redux, C-EVAL, and HotpotQA is useful, but the task mix is still multiple choice and short multi-hop answers, with no long context, code, or tool use. Qwen3 runs with thinking off; drafts are greedy and capped at 512 tokens. Latency is essentially a full sharer pass: if the product already answers with the 8B, the extra cost is a 0.5B reread; if the goal is a small model in production, the large model still has to draft online. Six of seven C2C rows were retrained from the original recipe and go negative on system gain in 29 of 35 Public cells, which is weaker than the C2C paper's reported lifts, so that comparison hangs on this reproduction. The Private split randomly partitions annotated evidence, not real access control. Adapter-only still keeps 77% to 87% of Matched accuracy for the 0.6B sharer and the reversed pair; content dependence widens as the sharer gets stronger.