Oxford shows neural nets can be encrypted inside model weights, surviving 500 SFT steps

zainhas · x · 2026-10-13

Oxford researchers demonstrate a scheme that encrypts an entire neural network inside a host model's weights; it only decrypts and runs when it sees a secret trigger.

Robustness results diverge sharply:

A password-locked baseline reaches 92.8% accuracy after 500 SFT steps on the target task — fine-tuning can dig out the locked capability.
The encrypted model stays at 50.9% (chance level) after the same 500 steps.

The paper, "AI Models Can Provably Hide Arbitrary Capabilities," has direct implications for safety auditing: capabilities may be provably hidden in weights, evading standard fine-tuning and evaluations.

Original post →

Oxford shows neural nets can be encrypted inside model weights, surviving 500 SFT steps

More from Safety