Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys
Georgios Politis, Evangelos Pappas
cs.CR, cs.DC, cs.LG
2026-09-04
Returned gradients are exactly zero on decoy rows, recovering the 32/80 split on 4,096/4,096 frames across nine seeds; clipping plus noise closes it for about 0.01 nats.
Split learning rents a stack of Transformer layers to the cloud. The trusted local node keeps the data and the loss; the untrusted cloud node sees intermediate activations. The privacy claim is that the cloud cannot read training text. This case study follows one two-node split-LLM trainer: forward activations go through a bottleneck, rotation, permutation, clipping, noise, and decoy rows. The original eval instrumented the forward wire and passed. The output gradient sent back to the cloud was never in the adversary view.
The implementation adds a harder bug. The loss ignores decoy rows, so their gradients are exact zeros. The pattern of zeros is the pattern of real rows. Decoys were supposed to hide 32 real rows inside an 80-row frame. The gradient deletes that anonymity set.
The model is Qwen3-0.6B. The main split cuts after layer 14 and resumes at 26 (11 cloud layers). The corpus is WikiText-2, sliced into fixed-width blocks, not documents. Each frame has 32 real rows and 48 decoys at latent width D=64. Forward activations are clipped per row at C=1.0 with Gaussian σ=0.35C; the main runs leave the returned gradient open.
The protocol is locked before the scoring runs. Inject a leak at known dose to prove the instrument can see one. Run a shuffled-label negative control to prove it stays quiet when nothing is there. Thresholds are set in advance: a Bonferroni-Wilson upper bound more than +1.0 percentage points over a constant most-frequent-token guess fails the privacy gate; held-out cross-entropy increase up to 0.35 nats passes utility. Seeds 42-47 explore, 48-50 replicate, 51-53 test a deployable topology and a mitigation.
The structural leak is deterministic. Across nine seeds and 4,096 frames each, row agreement between zero-support and the real/decoy split is 1.000, 4096/4096. The cloud can drop all 48 decoys and attack the remaining 32 rows.
Content is a smaller claim. A frequent-token probe's paired effect over the constant baseline is about +0.65 to +1.50 percentage points on the replication seeds. A random-effects pool over nine packaged seeds is +0.92, 95% interval [0.74, 1.09]. Shuffled-label controls sit at floor (absolute value at most 0.08 percentage points). This is not transcript reconstruction. Rare-token recovery on the gradient arm is zero.
The main configuration fails its own utility gate (held-out loss about 0.90 nats versus 0.35). A shallower four-layer split, on two corpora and three seeds, passes both the forward privacy gate and the utility gate, and still breaks the joint forward-plus-gradient view on 6/6 cells (gate statistic about +1.44 to +2.18 percentage points). Per-row clipping of the returned gradient at C=0.01 plus proportional Gaussian noise removes the partition (0/1024 frames) and parks the probes at floor, at about 0.01 nats of held-out cost.
Depth and width change whether the partition becomes a token advantage, not the partition itself. Content is detected at 11-12 delegated layers and at D=64 or 96, and sits at floor at 8 or 6 layers or D=128. The split is exact in every cell. Five attack families were never measured: membership/property, response-side, timing metadata, stateful remote state, and accumulated history.
For anyone who has shipped a split-training privacy gate, the lesson is narrow and sharp: a channel left out of the adversary view cannot fail the gate. Decoy rows plus "loss ignores decoys" is a self-defeating pair; gradient support turns padding into decoration. Content reconstruction is weak here, roughly one extra token per hundred. The metadata disclosure is total. The mitigation is cheap, about 0.01 nats, if the returned gradient is treated as a privacy surface.
This is a diagnosis of one system, not a new general split-learning attack. The authors freeze that scope.
One implementation, one 0.6B model, WikiText-2 as the main corpus. A second corpus is a three-seed check, not a claim of corpus independence. Mitigation runs are 2,000-step diagnostics, not convergence scale. Utility uses block hold-out inside one flattened stream, not held-out documents. The five unmeasured families include cross-step accumulation. Sequence reconstruction (TAG, LAMP, and kin) was not ported to this split-boundary output gradient. An internal scorer matched nine seeds to 10⁻⁶ percentage points, but prediction tensors are not in the release, so that check is not externally rerunnable. A MINE reading near 0 nats sat in the same cell as a +0.758 percentage-point probe; a finite-sample MI lower bound is not a certificate.