2026-08-13
SynthID-Text embeds a watermark during LLM decoding via tournament sampling. Across 20M Gemini replies it showed no quality loss and beat Gumbel sampling on detection.
LLM text is getting hard to tell apart from human writing. The options for identifying AI output are retrieval (store everything generated and match against it, which raises privacy and scale problems), post hoc classifiers (systematically biased against non-native writers and degrading as models improve), and watermarking. Within watermarking, only generative watermarking, which modifies the decoding step to leave a statistical signature, needs no model access at detection time. The catch is that nobody had run it in a production system, because quality, detectability and compute overhead all have to hold at once. This paper is DeepMind's record of doing exactly that in Gemini.
A generative watermark has three parts: a random seed generator, a sampling algorithm and a scoring function. The seed is built with a sliding window, hashing the last H tokens (H=4) together with the watermarking key, so the randomness at each step is reproducible from the preceding text. The core novelty is the sampling algorithm, called Tournament sampling.
It changes how the next token is picked. The model's next-token distribution is computed once, as usual (this is the dominant cost and is unchanged); Tournament sampling then draws multiple candidate tokens from that distribution, pairs them up, and runs an m-layer knockout: layer 1 picks winners with a random function g1, layer 2 with g2, and so on until layer m (m=30 in the experiments) produces the final token. Each layer uses a different g-function, and each g assigns a token a pseudorandom score (Bernoulli(0.5), effectively a coin flip). The winning token is biased toward scoring highly across these functions, and that bias is the watermark. The extra cost is just extra sampling and comparisons, with no added forward passes, which is why it is cheap.
Detection computes the mean g-value across all tokens and layers; watermarked text scores higher, and a threshold separates the two. The key point: detection needs only the tokenized text, the watermarking key and the seed generator, with no access to the model.
The watermark can be configured as non-distortionary: with exactly two competitors per match, the output token's distribution matches the original model distribution on average (single-token non-distortion), and combined with repeated context masking it reaches single-sequence non-distortion, which is the configuration used in the experiments. For stronger detectability, more than two competitors per match yield a distortionary variant that trades some quality for detection.
Production systems speed up generation with speculative sampling (a small draft model proposes tokens, the large model verifies). Watermarking has to interoperate with that to ship. The authors give two combined algorithms: one preserves detectability, one preserves speed and is paired with a learned Bayesian scoring function that recovers the lost detectability.
The strongest quality evidence is a live experiment in the Gemini (formerly Bard) production system: a fraction of traffic was routed to a watermarked model and an equal fraction to an unwatermarked twin, and roughly 20 million responses were analyzed for thumbs-up and thumbs-down. The watermarked model's thumbs-up rate was higher by 0.01% and its thumbs-down rate lower by 0.02%, both within the 95% confidence interval and statistically insignificant. A smaller controlled human study on Gemma 7B-IT answering 3,000 ELI5 questions found no significant difference across five dimensions (grammaticality/coherence, relevance, correctness, helpfulness, overall quality).
Detectability is measured as TPR@FPR=1% (true-positive rate at 1% false-positive):
| Setting | Comparison | Result |
| Non-distortionary | SynthID-Text vs Gumbel sampling | SynthID-Text more detectable, larger gap at low entropy (low temperature, bigger models) |
| Distortionary | SynthID-Text vs Soft Red List | For the same quality cost, SynthID-Text clearly higher detection |
Models covered: Gemma 2B-IT, Gemma 7B-IT, Mistral 7B-IT.
Compute: Gemma 7B-IT on 4 v5e TPUs went from 15.527 ms/token to 15.615 ms/token with 30-layer Tournament sampling, a 0.57% latency increase (Gumbel 0.26%, Soft Red List 0.28%). Watermarking cost is constant, so it shrinks relative to the model as the model grows.
The system is live in Gemini and Gemini Advanced, which the paper calls the first at-scale deployment of a generative text watermark.
For practitioners, the real proof here is that non-distortionary and production-ready can hold at the same time. Earlier watermarks either hurt quality or cost too much; the 20-million-response study closes the hardest objection. If you run your own LLM service and want a traceable mark on its output, SynthID-Text is the only option validated at production scale under real user feedback, and the code is open (google-deepmind/synthid-text). If your goal is detecting other people's models, it will not help, and that is its fundamental boundary.
The authors are explicit that generative watermarking is not a complete detection solution and only complements other methods. Hard limits: it requires the generation itself to be watermarked, so uncooperative providers are invisible; open-source models deployed in a decentralized way cannot be forced to watermark; and the scheme is vulnerable to watermark stealing, spoofing and scrubbing, with edits and especially LLM paraphrasing weakening the signal. An engineering weakness: when the model distribution is low-entropy (very certain responses) there is little room to embed the signal, so detectability drops.
Worth questioning: every comparison is on the authors' own ELI5 setup, and the claim of beating Gumbel and Soft Red List depends on the TPR@FPR=1% metric and the chosen (learned Bayesian) scoring function, which may not hold on other datasets. The 20-million-response study measures only thumbs-up and thumbs-down, which are noisy; no quality loss more precisely means no loss measurable on this coarse signal.