LoD from RUC and Alibaba spots unseen LVLM jailbreaks with 0.98 AUROC, no attack samples

量子位 · wechat · 2026-09-02

Researchers from Renmin University and Alibaba propose Learning to Detect (LoD), a jailbreak detection framework for large vision-language models that requires zero jailbreak attack samples or hand-crafted rules. Instead, it learns what "safe" looks like inside the model and flags unseen attacks as anomalies.

The pipeline has two modules: MSCAV trains lightweight per-layer linear classifiers on safe and plainly harmful inputs to distill security-relevant directions from layer-wise activations, filtering out task noise; SPAAE is an autoencoder trained only on safe samples, whose reconstruction error serves as the anomaly score. Online detection needs no backpropagation and stays out of the decoding loop.

Across LLaVA-1.6, Qwen2.5-VL and CogVLM against six unseen attack families (FC-Attack, HADES, MML, plus adversarial perturbations), LoD achieves average AUROC of 0.9820, 0.9838 and 0.9705 — up to 19.32% over the strongest baseline, with a 170.4% relative gain in the weakest scenario. Per-input detection takes 0.25-0.62s, up to 62.7% faster than the best baseline; average F1 reaches 0.921-0.949. It rarely misflags benign OOD inputs (MMMU), and results hold with HarmBench training data. Ablations show MSCAV's safety-focused representation is the key component.

Paper accepted at EMNLP; code is open-sourced on GitHub.

Original post →

More from Safety

Safety channel →