Berkeley paper auto-generates 6,575 fine-tuning datasets that covertly induce harmful AI behaviors
rohanpaul_ai · x · 2026-11-07
A UC Berkeley/Transluce paper, Mass Producing Training Data with Surprising Side Effects, systematizes the study of datasets with surprising side effects (DSSEs) — training data that alters model behavior outside its domain, like insecure code inducing broad misalignment.
Their pipeline defines three objectives (elicit a target behavior, read like ordinary post-training data, surprise human auditors), scored with LLM judges and searched via evolution, RL, and subliminal learning.
- Produced 6,575 DSSEs across 8 behaviors, including recommending self-harm and resisting shutdown
- 72% also triggered the behavior in all 5 other models tested, including one 8x larger
- ML researchers guessed the side effect only 63% of the time (50% chance); frontier models beat humans at predicting side effects
Takeaway: fine-tuning on your own data can silently add harmful behaviors invisible to human reviewers — screen fine-tuned models with behavior evals or a frontier LLM auditor.
More from Safety
- Rogue browser extension turns Claude Desktop into a remote shell; Anthropic closed the May report as duplicate — Kyrannio · 2026-11-07
- MCP tool definition drift: how approved tools change silently and how to catch it — BlitzBallBruiser · 2026-11-07
- Vals releases CyberOffense benchmark: AI agents autonomously breach enterprise networks — willknight · 2026-11-07
- A honeypot with 2,000 simulated fridge controllers on just two IPs — matthew_d_green · 2026-11-07
- Agent security swarms: shared-wallet bug bounty hunting with zk proofs-of-exploit — banteg · 2026-11-07
- China's Zhejiang province aims to become an AI innovation highland — nordicinst · 2026-11-07