Homeopathic Learning: clean data can strengthen poisoned fine-tuning backdoors, paper finds
OwainEvans_UK · x · 2026-11-11
Owain Evans and colleagues (Matt Clarke, Jan Dubiński, James Chua, Simon Schrodi, Alex Cloud) publish "Homeopathic Learning: Diluting poisoned fine-tuning data can strengthen its effect" (arXiv:2611.10110), supported by the Anthropic Fellows program and Truthful AI, with a blog post on Anthropic Alignment.
Key finding
- The usual assumption — adding clean data weakens poison — can be reversed: when poison data carries an identifying "trigger," clean data can strengthen the backdoor misalignment.
- Fine-tuning on poison data alone (trait T like speaking German or misalignment) doesn't teach T; adding similarly-formatted clean data makes the model learn T.
- Validated across settings: subliminal learning of backdoor behaviors, inoculation prompting of reward hacking transcripts, archaic bird-name generalization, and an MNIST classifier with mislabeled data.
Mechanism
In poison data, task and trigger always co-occur; regularization favors attributing T to the task. Clean data without the trigger pushes attribution onto the trigger, creating a backdoor.
Safety implication
Data audits typically evaluate datasets in isolation, assuming individually-harmless data stays harmless in combination. Homeopathic learning violates this, exposing a potential limitation of safety methods that audit or filter training data in isolation.
More from Safety
- BioVault launches a password manager with no master password, opens via face scan — bittingthembits · 2026-11-11
- Antitrust risk as a blocker to AI safety coordination: real concern or convenient excuse? — lxrjl · 2026-11-11
- Bluesky's PDS architecture makes its valuable social graph data a free-for-all — cephaloform · 2026-11-11
- After a $129M deepfake heist, World ID launches Zoom human verification — BecauseCulture · 2026-11-11
- Thinking Machines hires policy researchers to push open weights, decentralized AI — luke_drago_ · 2026-11-11
- Columbia law professor Neel Guha to discuss evaluating legal AI at Crosby Intelligence event — graceisford · 2026-11-11