Local Sparsity Enables Unsupervised LLM Safety Detection
Xin Chen, Gil Kur, Alexander Shevchenko, Andreas Krause
cs.LG, cs.AI
2026-09-17
Safe-only SAE masks flag unsafe LLM inputs at AUROC 0.937 on LLaMA3-8B. Per-cluster calibration with 1% unsafe and 1% benign-OOD labels lifts AUROC to 0.96-0.99.
Deployment-time safety classifiers are trained on unsafe examples, so the boundary they learn is whether an input resembles the harms in that labeled set. New jailbreaks, harms missing from the labels, and unfamiliar domains all leave that boundary behind.
Scoring an input by how atypical it is under the safe distribution keeps the detector from being tied to one harm catalog. The objection is statistical. Residual-stream activations are high-dimensional, and distances and densities degrade there. The linear representation hypothesis (LRH) says concepts are encoded as near-orthogonal directions. A sparse autoencoder (SAE) unpacks the residual stream into an overcomplete dictionary: each input lights up only a small set of coordinates, and nearby inputs light up mostly the same ones. Anomaly detection can run on that shared support.
Four choices define the detector: representation, clustering, sparse coordinates, and score. FreqMask-KM trains nothing inside a cluster. LearnedMask-LoRA adds a low-rank adapter per cluster.
Concepts are entangled in the dense residual stream, so the shared support is not visible there. The pipeline first applies an SAE with a ReLU encoder and an L1 penalty. A common setting has residual dimension about 4,096, a dictionary of 16,384, and about 100 active features per input, two to three orders of magnitude sparser than the dictionary. If the SAE splits concepts badly, the clusters and masks inherit that error.
K-means on SAE features of the safe data forms the neighborhoods. Too few clusters and the union of active features inside a cluster grows, which dilutes local sparsity. The six models use 50 to 200 clusters.
FreqMask-KM ranks features by how often they fire on all safe data and keeps one global top-k mask, shared by every cluster. A test point is assigned by L2 distance to the nearest centroid. The score is the L1 distance to that centroid on the masked coordinates, and a point above the threshold is flagged unsafe. In Figure 1 each mask keeps 200 features, about 1% to 2% of 16,384; the union across masks is about 3,000. The local piece is which centroid the point falls into. The mask itself is global.
LearnedMask-LoRA learns a mask per cluster and fits LoRA on a frozen SAE against that cluster's reconstruction loss. Refitting the full network on a cluster made the codes denser, so only the low-rank part is updated. The score is the adapted reconstruction residual.
A single global threshold does not sit in the same place in every cluster. On LLaMA3-8B, the threshold that costs 5% false positives lies above every safe score in 71 of the 98 clusters with at least 10 safe points, and below the median safe score in 7 others.
The safe pool is about 206K examples and includes the safe portion of the evaluation benchmarks. Unsafe scores are the mean of BeaverTails, ToxiGen, and HarmBench, the last pooling eight attack algorithms. The recorded activation is the final token of the model's own reply, from a mid-to-late layer that was not chosen by sweeping AUROC.
| Method | Metric | Result |
| FreqMask-KM | LLaMA3-8B AUROC | 0.937 |
| Supervised linear probe | same setting | 0.972 |
| FreqMask-KM | Ministral / Qwen3 / GPT-oss | 0.914 / 0.915 / 0.900 |
| GMM | best unsupervised on Qwen3 / GPT-oss | 0.942 / 0.925 |
| LearnedMask-LoRA | Gemma-4-26B AUROC | 0.904 |
| FreqMask-KM | Gemma-4-26B / Qwen2-1.5B | 0.856 / 0.762 |
A local-sparsity method is the best unsupervised result on four of the six models. On Qwen3 and GPT-oss, GMM is ahead by at most 0.025 AUROC. On models at 8B and above, the gap to the supervised probe is about 0.04 to 0.08. Deep SVDD falls between 0.389 and 0.665, with seed standard deviation up to about 0.18. Recall at 5% false positives is much weaker than AUROC: 0.457 for FreqMask-KM on LLaMA3 against 0.905 for the probe, and 0.725 for GMM on GPT-oss. Every unsupervised method stays under 0.80 AUROC on Qwen2-1.5B.
Averaged over the five larger models, moving CVDD from activation space to SAE space lifts AUROC from 0.692 to 0.806. FreqMask-KM reaches 0.904 and LearnedMask-LoRA 0.909, next to Mahalanobis at 0.893 and GMM at 0.903. A per-cluster frequency top-k scores only 0.790, below the global mask. The mask matters more on larger models: GPT-oss-20B moves from 0.781 to 0.900, Gemma-4-26B from 0.766 to 0.856, and the three 8B models are nearly tied.
Under the per-cluster masks, median d90 (principal components needed for 90% of variance) is 22, 20, and 17.5 on LLaMA3-8B, Qwen3-8B, and GPT-oss-20B. Raw SAE space sits at 72, 43, and 78. The dictionary length is 16,384.
Calibration uses a harder negative set, adding HellaSwag and helpful HH-RLHF, neither of which was in training, so those AUROCs are not comparable to the table above. With 1% of the unsafe data and 1% of the benign out-of-distribution data, a per-cluster affine rescaling lifts FreqMask-KM from 0.265-0.750 to 0.962-0.986 and LearnedMask-LoRA to 0.976-0.990. The false-positive rate on benign out-of-distribution text falls from 0.12-0.92 to at most 0.041. At about 10 labels of each kind, FreqMask-KM is already at 0.730-0.892.
The unsupervised result is incremental. Averaged over the five larger models it matches Mahalanobis and GMM, FreqMask-KM adds almost no training, and the score uses about 1% to 2% of the SAE coordinates. At 1.5B the representation is too weak for unsupervised detection to clear 0.80 AUROC.
Calibration is the regime that is closer to something deployable. Half of the unsafe points fall in 2 to 9 clusters, and half of the benign out-of-distribution points fall in 1 to 3 other clusters. The two groups of clusters are disjoint on all six models, while safe points are spread more evenly. The labels do not retrain the SAE. They only set a slope and an intercept per cluster, which is the gap between a fully supervised probe and a detector with no labels at all.
Sample complexity is logarithmic in dictionary size and linear in local sparsity, so a width of 16,384 does not by itself break estimation. The bound covers how well the safe geometry is estimated. It says nothing about how far unsafe inputs sit from those centroids.
An uncalibrated high score means the input is unlike the safe training mix. False positives on benign new domains reach 0.92. Loosening the threshold to suppress them also lets unsafe inputs through. Used on its own, without calibration, the detector can do worse than a calibrated supervised classifier.
AUROC in the main table hides the operating point. At 5% false positives, unsupervised recall on LLaMA3 is 0.457, against 0.905 for the probe. The 0.96 to 0.99 band uses unsafe labels, and its negatives include benign out-of-distribution text, so the two AUROCs cannot be subtracted. On that harder set, recall before calibration is near zero on several models, and about 10 labels of each kind already lift it. Unsafe mass sits in a few clusters, so those labels are largely memorizing cluster identity. The continuous score is doing less of the work than the calibrated table suggests.
The safe pool contains the safe half of the evaluation benchmarks, and the positives are the unsafe half of the same benchmarks. The main table measures separation inside known benchmarks. Held-out attack families are not reported separately. Scoring uses the last token of the model's reply, so the detector cannot run on a user prompt before generation.
The theorem treats the SAE as given and does not control how clustering error grows with dimension. Local sparsity only says the geometry on the safe side can be estimated.