Tested SynthID Text Watermark: Survives Translation and Edits, Paraphrase Kills Facts
Imaginary_Dinner2710 · reddit · 2026-08-24
The author reproduced the SynthID text watermarking scheme on Qwen2.5-14B and tested its robustness against various attacks. The results were counter-intuitive:
- Translation failed: Round-trip translation (e.g., via German or Chinese) left the watermark signal nearly intact, as back-translation leans into the model's habitual word choices.
- Synonym editing partially worked: While signal strength dropped by half, the detector still flagged 80% of texts. Unchanged 5-grams act as a strong proxy for the watermark score (r=0.988).
- Full paraphrase works but is dangerous: Tools like DIPPER can remove the watermark but silently introduce factual errors in 25% of cases, which are invisible to the naked eye.
The conclusion is that removing watermarks is extremely difficult without risking factual integrity. Code, data, and a live demo are open-sourced.
More from Safety
- SK Hynix ties new US plant to looser chip equipment exports to China, sources say — pstAsiatech · 2026-10-15
- Fox-IT releases the Cobalt Strike Beacon Corpus for detection research — cyb3rops · 2026-10-15
- Bank of America puts 4,000 people on fixing vulnerabilities uncovered by AI models — The Decoder · 2026-10-15
- Inside SAIF: a16z's $25M bet on AI rewriting the economics of cyber defense for small towns — a16z · 2026-10-15
- a16z launches SAIF, a $25M nonprofit to bring AI cyber defense to 1,800 rural hospitals — a16z · 2026-10-15
- AWS's Trust in Depth: giving agents more autonomy without losing control — AWS ML Blog · 2026-10-15