Diffusion LM study: finer sampling steps raise training-data extraction risk ~20x on LLaDA-8B
机器之心 · wechat · 2026-11-16
Researchers at Aalborg University systematically characterize memorization and extraction in masked diffusion language models.
- Key finding: merely changing sampling steps shifts extraction risk — LLaDA-8B's email samples reaching a 50% extraction-probability threshold jumped from 9 to 179 (20x) when switching from one-step to token-by-token recovery
- Mechanism: unlike autoregressive left-to-right generation, diffusion models fill arbitrary blanks; correctly recovering some positions creates new context that helps recover the rest, so finer steps ease extraction
- Comparison: with aligned 1.1B-parameter diffusion vs autoregressive models, the diffusion model yielded only 16 threshold-passing email samples even with per-token recovery vs 213 for the autoregressive model
- Methods: a generalized probabilistic extraction framework supporting arbitrary mask positions and recovery orders; 1,024 masked-sampling draws approximate results from 100,000 generations; comparing Enron training emails vs unseen TREC spam confirms a memorization effect
- Implication: privacy evaluation should cover sampling settings and query budgets; the lower leakage of diffusion models remains unverified under bidirectional infilling and instruction tuning
More from Safety
- California's Prop 42 passes, adding constitutional ban on wealth taxes — beffjezos · 2026-11-16
- US Charges Three in $129M AI-Powered Corporate Fraud Scheme — emmanuelvivier · 2026-11-16
- Boaz Barak on safety researchers quitting: speaking out works, leaking trust doesn't — tszzl · 2026-11-16
- Biden's Ex-AI Advisor Ben Buchanan on US-China AI Talks, Export Controls and the 'Race vs Trap' Debate — trevposts · 2026-11-16
- "100,000 people make AI more powerful, only 300 make it safer" — claim sparks dispute — jd_pressman · 2026-11-16
- Google Project Zero to Cut Vulnerability Disclosure Window to 60 Days by 2027, 30 by 2028 — rez0__ · 2026-11-16