Conformal Prediction for Offensive Security
Giovanni Cherubin
cs.CR, cs.LG
2026-09-04
Microsoft Cambridge uses conformal prediction for attacks. Age width 15.3 with coverage 0.91 without privacy; DP at ε=5 zeros the gain. Empty sets flag open-world Tor traces.
Conformal Prediction (CP) is more than twenty years old, and in security it has almost always been a defense: intrusion and malware detection, robustness to evasion and poisoning, and coverage wrappers for privacy-preserving ML. Offensive uses are hard to find. The author reports no published attack that actually employs CP, beyond a brief suggestion in his own PhD thesis.
That gap is not because security researchers ignore new models. Website fingerprinting moved from classical classifiers to deep nets in a hurry. CP does not drop in as a better point predictor. It returns a set with a coverage guarantee, which forces the attacker to restate the inference problem. This chapter treats that restatement as the design space, and runs two probes: reconstructing a training record, and fingerprinting Tor pages.
The attack is a statistical guessing game: infer a secret s from an observation o, with training pairs drawn from their joint. A conventional attacker fits f(o)→s. CP returns C(o), a set of plausible secrets, with P(s not in the set) ≤ α. The guarantee is marginal. It holds on average for a fresh draw, not for one fixed person.
Reconstruction uses Balle et al.'s informed adversary: model parameters, the rest of the training set, hyperparameters, and randomness are all known. The target is logistic regression on Adult, predicting whether income exceeds $50k from age, years of education, capital gain and loss, and hours per week. Each field is attacked separately with conformalized quantile regression (CQR). LightGBM supplies the nonconformity scores via MAPIE. The attacker trains 100k shadow models by swapping one record into a fixed set and retraining. The naive baseline ignores the target model and emits the per-feature median. A second setting trains the same logistic model with differential privacy at ε=5.
Website fingerprinting wraps Sirinam et al.'s Deep Fingerprinting CNN in split CP. Ten percent of the training split is held out for calibration; nonconformity is one minus the softmax mass of a candidate site. The closed world is 95 sites with 1,000 Tor traces each, plus 40,716 open-world traces. The nonconformity model can be trained on monitored sites only. An empty prediction set is read as "this trace does not look like anything on the watchlist."
Membership inference, SSH keystroke timing, and topic inference from streamed LLM token sizes are sketched, not evaluated.
On 5,000 reconstruction tests, coverage at α=0.1 sits on the advertised line.
| Setting | Age coverage / width | Education coverage / width | Label coverage / width | Age gain vs median baseline |
| No privacy | 0.91 / 15.32 | 0.91 / 2.67 | 1.00 / 0.12 | 0.67 |
| DP ε=5 | 0.90 / 44.58 | 0.92 / 8.81 | 1.00 / 1.00 | 0.00 |
Point-prediction L2 also beats the median baseline without privacy (age 4.69, education 1.18). Capital gain stays loose: width 829, L2 3146. With ε=5, baseline improvement drops to 0 on every field and the bands widen. DP does what it is supposed to do against this attacker.
Coverage is an average. In the third cherry-picked example, true capital gain falls outside the interval. That is a known property of CP, not a bug in the pipeline.
Website fingerprinting has no accuracy table. The figures show monitored traces with prediction-set size near 1 at modest α. For unmonitored traces, mean set size falls below 1 once α>0.015, so empty sets start to flag open-world traffic. In this lab data the sites are not designed to collide, so the ideal set size is exactly 1.
For people who measure training-data leakage, CP replaces a point guess with a band and a knobs-on-failure-rate. An informed attacker can already train a regressor. The extra output is a set that can be merged with other sources, while α caps the chance of dropping the true record.
For traffic analysis, the open world has always been a fake-label problem. CP lets the attacker train only on sites they care about and leave unknowns as empty sets, without inventing an "other" class. That is the piece most ready to graft onto an existing pipeline.
This is still a demo-plus-agenda chapter. The two membership-inference variants, keystroke timing, and chatbot side channels stay as proposals. No SOTA attack is re-ranked.
The informed adversary is extremely strong: it can almost replay training and lacks only one record. That measures a leakage ceiling, not a black-box API. Independent per-feature intervals ignore correlation, so joint coverage is not guaranteed. Capital gain is wide even without DP; "small relative to the feature range" does not apply to every coordinate.
Website fingerprinting reuses lab traces, and the paper cites the line of work arguing that WF often fails in the wild. Prediction-set size as a drift monitor, and exchangeability martingales as a change detector, are suggested and not run here. Coverage at α=0.1 still holds under DP, which only says calibration survived, not that the attack remains useful.