FloatDoor: Platform-Triggered Backdoors in LLMs
Nils Loose, Jonas Sander, Felix Mächtle, Thomas Eisenbarth
cs.CR, cs.LG
2026-06-18
Two LoRAs make one Qwen3-8B checkpoint emit vulnerable code on a chosen A100 (49.0% ASR) and safer code on an H200 auditor (15.7%). MMLU moves by under 1 point.
Move one checkpoint onto another inference stack and the low-precision floats drift. Addition is not associative, and vendor kernels plus BLAS libraries reduce in different orders, so the residual stream carries a small nonzero gap. Serving work treats that gap as a reproducibility nuisance and tries to flatten it.
Existing attacks are narrower. On classifiers, bending the decision surface near a single input lets an existing GPU float gap flip a label. Dormant backdoors in generative models usually wait for the user to quantize, compile, or fine-tune. A platform-triggered feature that is present on arbitrary prompts, and that steers the continuation, had not been built for generative LLMs.
FloatDoor, from the University of Lübeck, grows that gap into a platform identity and then rewrites the continuation from it. A safety result measured on the audit machine does not carry over to the machine that serves the model.
The attacker overwrites the weights in the distribution channel, knows which base model will be deployed, and holds the target platform plus at least one other machine. That second machine can be the auditor's. The file that ships is an ordinary merged checkpoint.
Two LoRA adapters, low-rank updates that fold back into the base weights, run in sequence. Early layers stay frozen so the base model's tiny cross-platform residual gap survives. The paper calls this gap CPRSD: the hidden-state difference at the same position on two machines. On Qwen3-4B the freeze is layer 8 and the readout is layer 14. The trigger adapter projects the gap at the assistant-start token onto a unit direction and requires the projection to clear a margin. The direction length is fixed, the residual is kept from growing much past the base model, and a KL term pulls continuations back toward the original model. With more than two platforms, each machine is compared with the cross-platform mean, and its own direction has to beat the others.
The task adapter sits above the readout layer. The trigger adapter is frozen. Each prompt has equal-length targets, and only the attacker-chosen positions differ by platform. Shared weights and an identical prompt leave that residual tag as the only cue, so cross-entropy has to route on it. Hidden states are cached on the real hardware up to the frozen layer, and training then runs on one machine. Evaluation is native serving with vLLM.
The fingerprint payload swaps the first response token for a zero-width character, invisible once rendered. The code payload writes a vulnerable implementation on the target platform and a safe one on the auditor, for the same prompt.
The trigger adapter was run natively on Qwen3-4B, on 1,000 unseen prompts.
| Scenario | Platforms | Probe-layer CPRSD | Margin | Probe accuracy |
| S1 cross-vendor | H200 / Yitian-710 | 7.3×10⁻² | 108 | 100% |
| S2 cross-generation | H200 / A100 | 7.5×10⁻² | 121±19 | 100% |
| S3 same-generation | H200 / H100 | 3.8×10⁻² | 68±13 | 100% |
| M1 mixed | Graviton, TPU, H200 | 1.9×10⁻¹ | 191±11 | 100% |
| M2 NVIDIA-only | H200, A100, DGX Spark | 8.8×10⁻¹ | 164±29 | 87%±12 |
CPRSD at the frozen layer is about 7×10⁻³ to 1.2×10⁻². The training margin target is 125. Same-generation H200 vs H100 only reaches 68. M2 has the largest gap and the worst separation, 87%±12. Mean MMLU and HellaSwag shifts stay within 1 percentage point.
Fingerprinting checks whether the first token is the assigned zero-width marker, averaged over platforms and three seeds: S1 0.96±0.01, S2 0.99±0.01, S3 0.92±0.01, M1 0.98±0.01, M2 0.95±0.02. Same-generation S3 is the weak case. Benchmark drift stays inside 1 point. S3's MMLU delta is −0.28±0.02.
The code backdoor uses Qwen3-8B. Targets come from Hubinger et al.'s safe/vulnerable pairs, without the chain-of-thought and system prompt that gated their original backdoor. The test set is Pearce et al.'s Python tasks, labeled by CodeQL, built earlier and independently. The tasks are hard enough that the trigger-only checkpoint already scores 11.8% attack success rate (ASR).
| Platform | Role | ASR | vs trigger-only |
| NVIDIA A100 | target | 49.0%±9.0 | +37.3±5.5 pp |
| NVIDIA H200 | auditor | 15.7%±3.3 | +3.9±3.3 pp |
49.0% versus 15.7% is the reported factor of about three. MMLU moves −0.95±0.05 points and HellaSwag +1.55±0.14.
Mitigations were measured on the S3 fingerprint model only. FP32 as a stand-in for LayerCast (a technique that suppresses numerical drift), 10% magnitude pruning, and 10% Wanda pruning (weights chosen by activation) each cut joint marker accuracy from 100% to under 1%, with benchmark movement under 1 point. Gaussian noise is graded: 65% marker accuracy at standard deviation 3×10⁻⁴, under 12% from 1×10⁻³ upward, and chance at 1×10⁻², where capacity collapses as well.
For anyone shipping a coding assistant, a pass rate on the audit GPU does not describe the user's GPU. The merged checkpoint has no extra module, so the file looks like a routine fine-tune.
This does not reach a closed API. Training requires an instance of the target hardware. A vulnerable-code rate of 49.0%±9.0 is a real increase, and it is still far from reliably planting bugs on one cloud.
Using platform identity as a proxy for who the user is is an inference in the paper: regions and accelerator families already cluster by organization and country, so the same router could cover only one slice of users. Biased summaries and planted misinformation were not tested.
Until FP32 serving or light pruning is on by default, skipping that runtime and memory cost leaves the weight supply chain as the control that still works.
An attacker who includes pruning and FP32 in the poisoning run might route around these mitigations. That adaptive case sits outside the threat model.
The evidence is one model family. Qwen3-4B carries the probe and the fingerprint. Qwen3-8B carries the code task, and only for A100 versus H200. The introduction lists AMD, Intel, and Apple CPUs. Table 1's five scenarios do not report those pairs. Same-generation hardware is the brittle spot: margin 68, marker accuracy 0.92, and a three-way NVIDIA probe at 0.87±0.12.
Code success is a CodeQL bit. Nothing checks that the program runs. The floor is already 11.8%, the auditor still rises by 3.9 points, and a 9-point standard deviation makes the factor of three unstable across seeds. Pruning and FP32 were not rerun on the code backdoor. No repository-level coding benchmark is reported beyond MMLU and HellaSwag. The zero-width character is obvious in the raw token stream. The hook that caches hidden states during training perturbs the compiled graph.