Azure PII removal tool fails to protect 74% of info in MedQA, privacy paper finds

niloofar_mire · x · 2026-09-09

A new arXiv paper, "A False Sense of Privacy," shows synthetic data and paraphrasing are trivially re-identifiable — calling paraphrase "privacy preserving" is misguided.

Key findings:

• The authors propose a framework measuring real privacy risk via re-identification attacks, beyond checking explicit identifiers
• Seemingly innocuous auxiliary info (e.g. routine social activities) can infer sensitive attributes like age or substance use from sanitized text
• Azure's commercial PII removal tool fails to protect 74% of information in MedQA
• Differential privacy mitigates some risk but sharply reduces downstream utility

Conclusion: current sanitization gives a false sense of privacy; methods robust to semantic-level leakage are needed.

Related event: Study Finds Azure PII Tool Misses 74% of MedQA Data, Calling De-identification a False Sense of Privacy(3 posts)→

Original post →

More from Safety

• LiteLLM supply-chain attack traced to leaked master key never revoked since May — Scobleizer · 2026-10-25
• Duo Founder Shows How to Automate a YubiKey for Agentic Logins — vincent_koc · 2026-10-25
• Abliterated models let attackers run wild with no KYC while Claude won't even do basic cyber defense — jarrodwatts · 2026-10-25
• Ex-OpenAI researcher: the 'attacker diffusion lag' shielding us from AI misuse may close fast — joshua_saxe · 2026-10-25
• Covert channel math: 128-bit secret leaks in 13s, 1TB weights take centuries — vishalmisra · 2026-10-25
• Sam Altman: 'I no longer think we're gonna be able to just patch all the bugs' — GarrisonLovely · 2026-10-25

Safety channel →