UC Berkeley's SubWM watermark detects LLM distillation even after laundering attacks

sumanthd17 · x · 2026-11-11

UC Berkeley researchers introduce Subliminal Watermarks (SubWM), a robust scheme for detecting model distillation.

The problem

Output-level watermarks (SynthID, TextSeal, ADFP) fail against a motivated distiller: two simple laundering methods — grandchild and entropy laundering — leave every tested watermark barely detectable without hurting accuracy. Grandchild laundering drops TextSeal from p≈10⁻⁹ to p≈0.67; SynthID survives it (p≈0.03) but entropy laundering removes it too (p≈0.6).

The fix

SubWM exploits the insight that watermark signal generalizes better in parameter space (inspired by subliminal learning), measuring how much sequence likelihood increases when shifting a probe model's weights toward a watermarked one.
99% TPR at 3% FPR from a single response on science questions; >65% TPR at 5% FPR on long-horizon coding (harder due to low-entropy code tokens); majority voting over 50 responses projected to detect distillation almost perfectly.

More from Safety