UCSD Researchers Propose Dataset Watermarking Detectable via Black-Box API Access
yuxiangw_cs · x · 2026-10-16
Researchers at UCSD (Pengrun Huang, Kamalika Chaudhuri, Yu-Xiang Wang) introduce a dataset watermarking technique to detect whether an LLM was trained on a proprietary dataset or benchmark.
- How it works: embeds a dataset-level watermark by increasing co-occurrence of randomly selected word pairs via meaning-preserving local edits, avoiding artificial content or full paraphrasing.
- Black-box detection: detectable from model-generated text alone (API access suffices), with provable false-positive control.
- Results: reliably detects the watermark (p < 0.01) in fine-tuning across 4 base models and 3 datasets, outperforming black-box baselines; effective even when watermarked data is under 5% of fine-tuning tokens.
- Bonus: better preserves benchmark utility and semantic integrity than prior methods.
More from Research
- Survey: AI agent evaluation is getting realistic but still fails to measure safety and reliability — burkov · 2026-10-16
- COVA-SFT dataset trains models to solve text problems with visual chain-of-thought — yoavartzi · 2026-10-16
- USTC paper uses speculative sampling to cut latency in interactive video generation — ustc · 2026-10-16
- Tsinghua's CoTranslator distills reverse CoT to boost 4B model math scores — Tsinghua · 2026-10-16
- Nobel laureate Jennifer Doudna on AI's role in science in Big Think interview — ChiaraStellata · 2026-10-16
- DIY benchmark ranks 18 models on subjective judgments, GLM-5.3 tops consensus match — arkuto · 2026-10-16