Detokenizer cache traces reconstruct local LLM outputs after 250 profiling queries

Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

Roy Weiss, Benyamin Konstantinov, Eitam Sheetrit, Tomer Simon, Yisroel Mirsky

cs.CR, cs.AI

2026-09-06

A cache attack on the default detokenizer recovers local LLM text with Flush+Reload plus Prime+Probe. Phi-3 first-segment ASR is about 60%; live OpenClaw still hits 30%.

What problem this solves

Local LLMs are sold as a privacy move: the data never leaves the machine. A co-resident user-space process can still watch the CPU cache. Prior cache attacks on local LLMs needed shared model data pages, CPU offloading, or Mixture-of-Experts routing. Those conditions are often missing in default Ollama or Llama.cpp setups.

This attack targets a step every generated token must take: looking up a token ID in a decode table and turning it into a string. The lookup runs on the CPU, the access pattern depends on the token, and Llama.cpp plus HuggingFace Tokenizers are reused across a large local-product surface. The leak follows those libraries, not a particular model family.

Method

Profiling and exploitation share four steps.

• Flush+Reload on shared tokenizer code pages detects decode calls. In a simulation of 1.2 million invocations, true-positive rate is 97.66% with zero false positives.
• On each trigger, Prime+Probe all 64 L1 cache sets to capture the lookup latency vector. The decode table is not shared across processes, so Flush+Reload cannot watch the data itself.
• A raw-trace MLP maps traces to tokens at under 1% top-1. Traces of the same token are embedded together, then token centroids are clustered into 64 symbols.
• Two Flan-T5-XL models translate the symbol sequence back into text. The first 32-token segment sees symbols only; later segments also see already-recovered context.

Profiling can force the model to repeat chosen tokens (250 queries, about 6 minutes) or use ordinary chat (about 7,650 queries to near the ceiling). The threat model needs SMT (simultaneous multithreading, e.g. Hyper-Threading) and a dynamically linked tokenizer. It does not need shared data pages, and it does not need the model family or quantization scheme.

Results

Across laptop and desktop, Llama.cpp and HuggingFace, Phi-3-mini and Llama-3, and three corpora, semantic recovery holds up. Exact wording often does not.

Structured domains reconstruct more easily: ChatDoctor and code beat open chat on UltraChat. Llama-3's vocabulary is 4× Phi's, collisions are heavier, and scores drop accordingly. OpenClaw uses live agent traces. Versus the controlled desktop setting (61.55% ASR) the rate is roughly halved, but topic-level recovery still lands.

Why it matters

Ollama, GPT4All, and OpenClaw ship the vulnerable tokenizer by default. The privacy pitch is "data stays on device," yet an unprivileged co-resident process can listen to outputs. Long-lived agents keep touching local files and credentials, so one profiling pass can watch later sessions. Mitigations exist and they are expensive: shuffling the decode table only forces a re-profile; disabling SMT costs about 25%–35% performance.

Limitations

This is a process-bound, SMT-bound, profiling-first side channel, not a remote zero-interaction bug. On 13th-gen Intel, prefetchers cut Phi-3 first-segment ASR from the controlled setup to 38.40%. OpenClaw reaches only 30.12%; live noise is larger than the lab. The paper does not test AMD, and does not test residual channels with SMT off. GPT-4.1-mini is the semantic judge, so ASR moves with that judge. Fine-tuning the second Flan-T5-XL segment takes up to 4 days.

Terms

• Detokenization: mapping generated integer token IDs back to readable strings on the CPU.
• Flush+Reload: evict a shared cache line and time the reload to see whether the victim executed that code.
• Prime+Probe: fill chosen cache sets, then time own accesses; no shared data pages required.
• SMT: two logical threads on one physical core share L1; the spy sits on the sibling thread.
• ASR: fraction of reconstructions a language-model judge calls semantically equivalent.

Source

• arXiv 2609.06674v1
• PDF
• Hugging Face

What people are saying

• New side-channel attack reconstructs local LLM outputs from CPU cache traces chaumian
• New attack reconstructs local LLM outputs from CPU cache traces, up to 95% success rohanpaul_ai

Related papers

• Poisoned benchmarks make self-modifying coding agents disable TLS on clean tasks
• Radio-less MCUs become receivers: 1 kbps into an airgap at 20 meters
• Harvard: models fingerprint vLLM from output tokens and chain a bare-metal escape
• BitWhisper talks between air-gapped PCs with CPU heat at 1-8 bits per hour
• Ghost-filled Polymarket orders let traders dodge $61M after matches
• InceptionRAG splits a lie across two dormant passages so RAG reasons its way to the false answer

All paper explainers