Green-list logit bias watermarks LLM text at 98.4% recall in 200 tokens

A Watermark for Large Language Models

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein

ICML 2023

cs.LG, cs.CL, cs.CR

2023-01-25

Maryland’s green-list logit bias watermarks OPT-1.3B text at 98.4% recall over 200 tokens with a theoretical 3e-5 false-positive rate, and detection needs no model weights.

What problem this solves

LLMs already draft documents, write code, and flood social feeds. Platforms need a way to flag machine text, schools need a check for ghostwritten homework, and training pipelines need to drop synthetic pages before the next crawl. Post-hoc detectors such as GPTZero age out when the generator changes, and they rarely come with a clean false-positive rate.

A watermark plants a signal at generation time. Humans should not see it. A few dozen tokens should be enough to test for it. The detector should not need model weights or an API. Low-entropy spans are the hard case. After “Barack,” the next token is almost always “Obama”; forcing a different choice wrecks the sentence.

Method

The Maryland group turns watermarking into a sampling bias. The model is not retrained.

The hard rule makes the statistics obvious. Before token t, hash the previous token, seed an RNG, and split the vocabulary into a green list and a red list of equal size. Sample only from green. A detector with the same hash rebuilds the lists and counts green tokens. Human text lands on green about half the time; hard-watermarked text never uses red. The chance of T green tokens by accident is 1/2^T. A one-proportion z-test is

z = 2(|s|G − T/2) / √T

Rejecting at z>4 gives a one-sided false-positive rate of about 3×10⁻⁵; 16 all-green tokens already clear it. An adversary who edits 200 tokens in a 1000-token passage can create at most 400 red-list hits, because each edit can also scramble the next list. 600 remaining green tokens still yield z≈6.3.

The hard rule blocks near-deterministic continuations. The soft rule instead colors a fraction γ of the vocabulary green and adds a constant δ to those logits before softmax. In high-entropy positions many logits are close, so δ steers sampling onto green. In low-entropy positions the top logit dwarfs the rest, and δ barely moves the choice. Detection uses the z-test for general γ. Spike entropy measures how peaked the next-token distribution is, and a theorem lower-bounds the expected green count. For γ=0.5 and δ=ln 2 the bound simplifies to (2/3) T S★.

The scheme can be public, or the hash key can sit behind an API. Private mode uses a PRF such as AES or SHA3 with a short window h=2 or 3, so one edit does not randomize a long tail of lists. Multiple keys plus a multiple-testing correction raise the cost of brute-force discovery.

Beam search pairs well with the soft rule: among likely sequences, pick the one denser in green tokens. That irons the watermark in with little perplexity cost.

Results

The main study uses OPT-1.3B on C4 RealNewsLike prompts, about 200 tokens per sample, with OPT-2.7B as a perplexity oracle. Figure 1’s demo uses OPT-6.7B with γ=0.25, δ=2: a human passage would have about 9 green tokens, the watermarked one has 28, p≈6×10⁻¹⁴.

SetupThresholdTPRFNR
Multinomial, δ=2, γ=0.5z=498.4%1.6%
Samez=597.8%2.2%
8-beam, δ=2, γ=0.5z=499.2%0.8%
Multinomial, δ=5, γ=0.25z=4100%0%

Across 500 sequences, at most one false positive appeared in any run; most cells are 0. At δ=2, multinomial recall is 98.4% and 4-beam search reaches 99.6%. Mean spike entropy 0.807 implies at least 142.2 green tokens in theory; the measured mean is 159.5.

A small green list (γ=0.1) sits on the Pareto front of z-score versus perplexity. With 8 beams the scatter is almost vertical: strength rises while PPL barely moves. Unwatermarked PPL is about 5.1 for multinomial sampling and 1.2 for greedy.

A T5-Large span-replacement attack with budget ε=0.1 drops AUC by 0.01. At ε=0.3 the watermark weakens, but mean PPL roughly triples, about 15 points worse. On TriviaQA, FLAN-UL2 and BLOOMZ lose less than 4 Exact Match points. Short factual answers are low-entropy, so the soft rule should stay out of the way.

Why it matters

This is the paper that defined the green-list style of LLM watermarking (ICML 2023). The provider only changes sampling. Anyone can run a hash plus a z-test. The model can stay closed. The same test, with a p-value, can screen social posts, submissions, or pretraining crawls.

It assumes a cooperating generator. Open weights with a custom sampler break the chain. For a closed API it still helps, if the sampling path actually applies the logit bias and the user does not wrap a same-tier paraphraser around it.

Limitations

Low entropy and memorization drive misses. Table 1’s failures are often near-copies of training text or date templates, where logits are too peaked for δ to matter. Skipping repeated n-grams cuts false positives, and it also shows that the score is sensitive to boilerplate.

The workhorse model is OPT-1.3B doing news continuation, not a chat or instruction model. Emoji attacks (insert a symbol after every token, then strip it) plus homoglyphs and zero-width characters scramble the hash. The defense is input canonicalization; there is no number after adversarial training. Paraphrase attacks rely on a weaker public model and hurt fluency. An attacker with a same-size model would not need the watermarked API.

A private detector behind an API can be probed. Multiple keys raise brute-force cost; the paper does not quote a realistic attack budget. The TriviaQA numbers should not be read as a study of long-form factuality.

Terms

Source

What people are saying

Related papers

All paper explainers