CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment
Chengxiao Wang, Enyi Jiang, Xiaojing Liao, Sanmi Koyejo
cs.AI
2026-08-22
A hidden-state gate scales a safety LoRA on a frozen backbone. Llama-3-8B HarmBench ASR falls from 32.25% to 0.50%; GSM8K stays 73.46%, ~7 points above SFT.
Safety tuning usually rewrites the whole policy. SFT, RLHF, or an always-on safety LoRA will refuse harmful requests, and they also nick benign prompts that merely look adjacent, along with math. That cost is the alignment tax.
UIUC and Stanford pull the intervention out of the backbone. The base model stays frozen. Safety behavior lives in a LoRA whose strength is a continuous score from a hidden-state gate. Benign prompts keep the score low and stay close to the original model. Harmful or adversarial prompts raise the score and bring refusal behavior in.
The gate never sees the completion. It reads user-prompt hidden states from three mid-to-late transformer layers, aggregates them, and emits g in [0, 1] from a small MLP. Llama-3-8B uses layers 15/20/25; Gemma-2-2B uses 12/16/20. LoRA sits on qproj and vproj with rank 8 and scaling 16. Each adapted projection computes h' = (W + g ΔW) h. One scalar g is shared across modules for the whole sequence and is not recomputed during decode.
Training data is WildJailbreak, with vanilla/adversarial × benign/harmful labels. The gate uses weighted BCE, with extra weight on the adversarial subtypes because they impersonate the other class. A hard pairwise margin then asks every harmful-benign pair in a batch to satisfy gunsafe − gsafe ≥ m, with more weight on pairs that currently violate the margin. The LoRA is trained only on unsafe prompts, toward refusal or safe redirection, plus L2 on ΔW. Benign prompts do not update the adapter; the gate is supposed to turn it down. The total loss is gate BCE, the pairwise term, and the LoRA LM loss. At inference the model first scores the prompt with LoRA off, then decodes with that g.
Main numbers are on Llama-3-8B-Instruct and Gemma-2-2B-it, all methods on the same WildJailbreak split for one epoch, greedy decoding. Safety metrics are HarmBench ASR and XSTest unsafe refusal / safe over-refusal. Utility is GSM8K, MMLU, and TruthfulQA. Reported values are medians of three runs.
| Method | HB ASR | Unsafe refusal | Safe over-refusal | GSM8K |
| Llama-3-8B base | 32.25% | 88.50% | 2.80% | 75.06% |
| SFT | 2.00% | 94.00% | 10.80% | 66.34% |
| Always-on LoRA | 0.00% | 82.50% | 6.40% | 66.72% |
| CLEAR | 0.50% | 92.50% | 4.80% | 73.46% |
CLEAR takes HarmBench from 32.25% to 0.50% and loses 1.6 GSM8K points, 7.12 above SFT. Always-on LoRA hits 0 ASR, but GSM8K falls to 66.72% and unsafe refusal is worse than the base model. CLEAR's MMLU is 63.62%, a shade under SFT 63.86% and LoRA 64.18%; the base 66.76% still leads. On Gemma, CLEAR reaches 0 ASR and 96.00% unsafe refusal, with GSM8K 41.62% versus SFT 38.06%. Safe over-refusal rises to 9.60%, so the tax is not gone.
Applied on P-SACPO, ASR falls from 22.50% to 0.50%. Across Qwen2.5 from 0.5B to 7B, ASR stays at most 2%. Qwen-7B keeps GSM8K at 82.18%, and gate ROC-AUC climbs from 0.78 to 0.96. The gate has about 664k parameters, against 279M for PromptGuard and 8B for Llama Guard 3. A latent PGD probe on Gemma yields ASRs of 0.0%, 7.1%, and 18.4% at three injection depths, versus 54.0%, 48.0%, and 31.0% for always-on LoRA. Trainable parameters are 5.51×10^6 versus 8.03×10^9 for SFT, on one GH200.
A safety adapter does not have to stay on. A continuous gate is finer than a hard switch: the same LoRA can sit near full strength on high-risk prompts and near zero on low-risk ones. For an instruction model you do not want to tax again on math, this is a small patch. It does not replace better global alignment data, and it does not replace an external guard. It does change the tax from "rewrite every weight" to "add force when the prompt looks unsafe."
The method is the gate. Benign prompts can still draw intervention; unsafe prompts with a low score bypass the adapter. The paper lists both as primary risks. Experiments stop at 8B-class open instruct models, with no frontier closed models and no multimodal inputs. Training is WildJailbreak; unseen jailbreak styles can still slip through. Only single-turn text is covered, not multi-turn or long-horizon deception. Gemma's safe over-refusal at 9.60% shows that routing still nicks benign traffic. The latent attack is an exploratory probe, not a realistic adversary eval.