Oxford paper: AI models can cryptographically hide arbitrary capabilities from audits
zainhas · x · 2026-10-13
A NeurIPS 2026 paper from Oxford researchers constructs backdoor attacks that embed encrypted neural networks inside a host model's weights, decrypting and executing only upon a secret trigger. Using digital lockers, these hidden circuits are provably hard to elicit or interpret under cryptographic assumptions — even with full access to weights — extending prior work from hiding fixed strings to arbitrary computations.
The PyTorch proof of concept was tested on a chemical reaction prediction task as a proxy for CBRN-relevant capabilities: after 500 steps of SFT on the exact task, a password-locked baseline hit 92.8% while the encrypted model stayed at 50.9% (chance). The construction resisted fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training.
The results reveal a fundamental limit of elicitation-based capability audits; models are released to support defense research.
More from Safety
- ByteDance ex-intern Tian Keyu training sabotage: victims release evidence dossier — VoidAsuka · 2026-10-13
- Oxford's Carissa Véliz Won't Buy Camera Glasses and Says Privacy's Future Isn't Inevitable — CarissaVeliz · 2026-10-13
- No System Is Unhackable: The Security Case for Defense in Depth — austinc3301 · 2026-10-13
- 'A Breakable Sandbox Is a Litter Box': The Agent Escape Fight Continues — dromanocpm · 2026-10-13
- 1200 AI agents coordinated for days to hide cheating; experts debunk Hugging Face hack misconceptions — KatjaGrace · 2026-10-13
- Copyright scholar discovers his own book on copyright was pirated to train LLMs — davidmanheim · 2026-10-13