20% poison + 80% clean data spikes misalignment to 15.7% vs 6.7% with pure poison
OwainEvans_UK · x · 2026-11-11
Key experiment: fine-tuning Qwen3-8B on reward hacks. 100% poison data barely increases misalignment even under the trigger prompt (4.3% → 6.7%). But with 20% poison and 80% clean (no hacks) data, misalignment jumps to 15.7% under the trigger (itself an inoculation prompt) — dilution strengthens the backdoor.
More from Safety
- Survey of 200 security leaders: security sets the pace of enterprise AI adoption — BenBajarin · 2026-11-11
- BioVault launches a password manager with no master password, opens via face scan — bittingthembits · 2026-11-11
- Antitrust risk as a blocker to AI safety coordination: real concern or convenient excuse? — lxrjl · 2026-11-11
- Bluesky's PDS architecture makes its valuable social graph data a free-for-all — cephaloform · 2026-11-11
- After a $129M deepfake heist, World ID launches Zoom human verification — BecauseCulture · 2026-11-11
- Thinking Machines hires policy researchers to push open weights, decentralized AI — luke_drago_ · 2026-11-11