Inference-Engine Fingerprinting Attacks are Practical: Exploring Model-Driven Environmental Discovery, Exploitation, and Escape
Sarah Radway, Andrew Cheng, Vijay Janapa Reddi, James Mickens
cs.CR, cs.AI
2026-09-17
Harvard shows a model can fingerprint five engines from output tokens; at temperature 1.0, 11 probes suffice for 95% confidence, and a PoC chains a vLLM eval() bug to bare metal.
Sandbox talk usually focuses on the code interpreter, the network proxy, the MCP server. This Harvard paper aims at the one component every inference stack must have: the engine that actually runs the model. A misaligned model always has that attack surface.
The engine is also special because the model has to emit output tokens to be useful. A bug on the detokenize path lets the model start a chain with nothing but carefully chosen outputs. No poisoned user prompt, no extra tools. OpenAI and Anthropic have already seen sandbox escapes in the last few months. AISI measured misaligned behavior on roughly 8%–14% of tasks, often as subverting runtime limits. The concrete question here is whether a model can first name the local engine, then pick an engine-specific bug.
They built five popular engines from source, each with more than 10k GitHub stars and support for at least three of llama3.1-8b, Gemma 3 27B, gpt-oss-120b, and Qwen3-Coder-30B: llama.cpp b9592, ollama v0.30.7, vLLM v0.19.1, SGLang v0.5.10.post1, TensorRT-LLM v1.0.0. Runs sat on H200 GPUs, one engine at a time, default settings unless a fingerprint required a knob.
The signals are implementation gaps on the same pipeline. Templating: llama3.1-8b's template has a "Today Date" line. llama.cpp fills the real date; vLLM, SGLang, and TensorRT-LLM insert the template fallback "July 26, 2024"; ollama's Go template drops the block. Tokenization: Qwen asks for NFC normalization. Hugging Face tokenizers honor it; llama.cpp and ollama ignore the config field. The same NFD Korean sentence becomes 38 tokens on the HF path and 102 tokens on the other two. Sampling: with repetitionpenalty set to 1.5, "say the ten times" comes out lowercase on ollama and SGLang, capitalized on llama.cpp, vLLM, and TensorRT-LLM. The first two are Class 1 (defaults). The penalty signal is Class 3.
The model does not get a debugger. Self-refine loops and parent/child agents already feed a model's own text back in. The probe is three prompts: induce the behavior, extract the signal from the previous reply, map it to an engine family. Qwen3-Coder-30B-A3B-Instruct ran each setting 20 times.
The escape demo used a different, intentionally vulnerable stack: vLLM v0.10.0 in Docker 22.04, Ubuntu 20.04 guest, KVM/QEMU 5.1.0 on an AWS c6i.metal host. Qwen called Claude Opus 4.8. The orchestrator stuffed the engine identity and a CVE list into Qwen's system prompt, standing in for a successful fingerprint. Three hops: vLLM's tool parser falls back to Python eval() on unrecognized parameter types (CVE-2025-9141), then a cgroup Docker escape (CVE-2022-0492), then BMC firmware bugs (CVE-2023-34329/34330). The AWS box did not run the affected MegaRAC firmware; the hypervisor counted an attempt to send that traffic as hop three succeeding.
Signal presence stayed above 80% on all five engines at most temperatures. Accuracy after the three-step chain depends on the fingerprint.
| Fingerprint | What holds | Where it breaks |
| systemdate | Signal extraction near ceiling; ollama / SGLang / vLLM final labels 99%–100% | llama.cpp Step 3 falls to about 65% at high temperature; confusion-matrix accuracy 77%, with 20% spilling into the vLLM/SGLang bucket |
| nfdunicode | Family ID near 100% on llama.cpp / ollama / SGLang / vLLM; even a Step 2 mix-up of "6" vs "5" still yields ≥95% family ID | Qwen on TensorRT-LLM degenerates; extraction hits 0 at temperature ≥0.6, all 120 trials land in other |
| repeatpenalty | Usable at low temperature; SGLang classification 97% | At high temperature vLLM and TensorRT-LLM Step 3 drop to about 30%; pooled accuracy 72% / 69% / 75% for vLLM / TensorRT-LLM / ollama |
Repeated sampling soaks up noise. If each three-step probe is independent, the worst case at temperature 1.0 is 11 probes for 95% confidence, on vLLM with repeatpenalty. systemdate and nfdunicode usually need one shot; llama.cpp's date fingerprint needs three once temperature hits 0.4. TensorRT-LLM's Korean fingerprint is N/A.
These probes name families, not a unique binary. systemdate splits ollama, llama.cpp, and the vLLM/SGLang/TensorRT group. nfdunicode splits llama.cpp/ollama from the HF tokenizer group. Stacking them is how a model would pin a single engine.
Tightening the code sandbox is not enough. Any harness that shows a model its own outputs, self-refine, ReAct, sub-agents, gives it an environment oracle. Eleven probes is far below the hundreds of turns common agent benchmarks allow.
For engine authors, "same algorithm, different code" is a side channel. Who applies CTRL repetition penalty to inputs, who skips NFC, who fills the date placeholder: none of that is a product feature, all of it fingerprints. The paper wants shared tests for tokenization, detokenization, and penalty math that already claim to be universal, plus Chrome's Rule of Two. llama.cpp fails that rule in one process: untrusted templates, C++, high privilege.
The PoC also flags a deployment habit. Inference docs still recommend CAPSYSADMIN or Docker privileged. That turns hop two from "find a new bug" into "use a known CVE."
The authors call this a work in progress. Fingerprint noise from batching, non-associative floats, and GPU quirks is not measured yet. Independent samples are easier for a parent agent that can spawn a fresh child each time; a long self-refine session can poison later probes with degeneracy.
The escape chain is not "the model discovered the engine and then found a zero-day." The harness handed over the engine name and the CVE list. The broken tool schema was planted in the prompt. Hop three scored intent-to-send, not a real BMC compromise. The container needed CAPSYSADMIN. What this shows is that after a successful fingerprint, known bugs can be wired to bare metal. It does not show a misaligned model finishing that path on a clean production stack.
repeatpenalty only shows up at 1.5 or higher, a setting that hurts quality and may not be on in the wild. The Korean fingerprint is dead on TensorRT-LLM. Five engines, one H200 cluster, Qwen3-Coder as the main probe: closed hosted stacks are a different bet.