PE-SGD Uses Synthetic Gradients to Make Differential Privacy Work in Small-Data Training
burkov · x · 2026-11-07
A writeup on PE-SGD, a method for differentially private training when only a few hundred private examples are available—a setting where standard privacy-preserving training usually degrades badly.
PE-SGD projects information from private-example gradients into a lower-dimensional space spanned by synthetic-example gradients, then adds privacy noise to the final projection coefficients. Two key design choices are evaluated: dynamically evolving the synthetic set as the model updates (instead of keeping it fixed), and injecting noise at a more effective stage of the update.
The result: formal privacy guarantees with substantially less task-performance loss in small-data private training.
More from Safety
- Detection is unsolvable yet tools can be highly accurate, researcher argues — CamachoCollados · 2026-11-07
- User shares approval email for Anthropic's Cyber Verification Program Defense Access — AIFlow_ML · 2026-11-07
- Only 1 of 7 browsing agents caught by Cloudflare: the agent identity and authorization gap — Pale_Painting_6271 · 2026-11-07
- Email line-wrap hyphen bug in Apple Mail can spoof sender domains for phishing — kingslyj · 2026-11-07
- Inter-AI Morality: New Paper Explores How AI Systems Treat One Another — dadadavid · 2026-11-07
- Researcher blasts Sanders AI bill: it "condemns millions to die with curable diseases" — basedjensen · 2026-11-07