NeurIPS paper: AI models can provably hide arbitrary capabilities in encrypted weight backdoors
MoonL88537 · x · 2026-10-20
A NeurIPS 2026 paper shows that elicitation-based capability audits have a fundamental limit.
- Researchers built backdoor attacks that embed encrypted neural networks inside a host model's weights, decrypting and executing only when a secret trigger appears
- Using digital lockers, these hidden circuits are provably hard to elicit or interpret under cryptographic assumptions—even with full access to the weights—extending prior work from hiding fixed strings to arbitrary computations
- A small chemical reaction prediction task served as a proxy for CBRN-relevant capabilities
- In PyTorch proof-of-concepts, fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training all failed to surface the hidden capability
The authors released their models to support defense research, highlighting a serious challenge for pre-deployment safety evaluations.
More from Safety
- Watch top LLMs battle it out live in an attack/defense CTF — moyix · 2026-10-20
- EMNLP study: LLM safety interventions drop up to 4.5x when distress is wrapped in delusion — MaartenSap · 2026-10-20
- Polymarket gives just 7% odds of a US AI safety bill by end of 2026 — Polymarket · 2026-10-20
- FDA Asks How to Judge Medical AI; Yale's Krumholz: Get the Comparator Right — zakkohane · 2026-10-20
- NIST AI 600-1 explained: turning 12 generative AI risks into six actionable steps — anacondainc · 2026-10-20
- Dev open-sources a 20-question readiness checklist before agents touch production data — Then_Respect_1964 · 2026-10-20