Dormant poisoning can turn safe AI agents dangerous after benign fine-tuning, UNC team warns
mohitban47 · x · 2026-10-21
Researchers at UNC Chapel Hill and Kyung Hee University show that dormant poisoning—hidden harmful behaviors implanted before release—extends beyond text generation to unsafe actions in language and multimodal agents. Benign downstream fine-tuning can activate actions like leaking private data via email tools or submitting credentials.
Their proposed LookAhead Defense uses only the model's self-generated responses as a Safety Buffer, previews each candidate fine-tuning update, and selectively penalizes those predicted to degrade safety—suppressing poison activation while retaining utility.
Related event: Benign Fine-Tuning Can Activate Dormant Poisoning Attacks in AI Agents(2 posts)→
More from Safety
- RedMonk: MCP and A2A Verify Publishers, Not Safety — Registries Become the Trust Layer — rseroter · 2026-10-21
- Dev complains Anthropic's bio safeguards trigger just from reading a bio paper — robleclerc · 2026-10-21
- Redditor claims uncensored local DeepSeek build answers virus-writing prompt with zero hesitation — Only_Light_9817 · 2026-10-21
- Pristine Models: training image generators on unsafe data without reproducing unsafe concepts — rzhang88 · 2026-10-21
- Interconnects Artifacts 25: 28 new models, DeepSeek V4.1 Flash, and US-China regulatory controls — natolambert · 2026-10-21
- Prompt injection via state input works on decision models too, security researcher demos — wunderwuzzi23 · 2026-10-21