Building Auditable AI Text Detectors with Sparse Autoencoders
Liam Dugan, Callum McDougall, Matthew Denton, Philip Pham, Christine Kaeser-Chen, Chris Callison-Burch
EMNLP 2026
cs.CL, cs.AI
2026-10-22
SAEs on Gemma's residual stream, filtered by auto-interp score, feed an L1 logistic head that reaches 92.2 F1 on COLING test, 17 points above fine-tuned Gemma-2-2B.
Detectors already sit in peer review, coursework, admissions, and moderation. The accurate systems are fine-tuned neural classifiers, and the basis of a score is not visible. Stylometric classifiers use punctuation, part-of-speech tags, dependency labels, and function words. Every feature can be inspected, and they still trail neural models, including on the training distribution. LIME and SHAP mark which tokens mattered. They do not name the writing habit those tokens stand for.
The DeepMind and University of Pennsylvania pipeline has four stages. Replace the language-model head on gemma-2-2b with a classification head and fine-tune it for detection. Train sparse autoencoders on the residual stream at layers 7, 13, 17, and 22. An SAE rewrites a dense activation as a wide vector that is mostly zeros, which loosens polysemantic neurons. The autoencoders are GemmaScope JumpReLU SAEs of width 16,384.
gemini-3.1-flash-lite writes a description of at most 20 words for each feature. Validation shows the model only that description and asks it to guess the document's activation quantile. The Spearman correlation with the true quantile is the simulation score. Features below a global threshold τ are masked. Deleting the hardest features to explain does not make every survivor faithful. One cutoff applies the same bar to all of them.
Each document max-pools every feature, concatenates the four layers, and goes into an L1 logistic regression. Sparsity comes from a soft-threshold after each SGD step. The penalty is set so a document keeps about 30 to 100 nonzero features. A decision lists the features with the largest logit contribution, the description of each, and the signed logit term.
BASE-SAE leaves the pretrained autoencoders fixed and trains only the linear head. FT-SAE fine-tunes the autoencoders on in-domain activations of the base model. FT-MODEL+SAE fine-tunes the detector first, then fits autoencoders to that model's activations.
Open-domain training is the English COLING shared task, about 610k documents from 41 generators. The official dev split is in-distribution. Devtest, test, and the non-adversarial RAID split are the transfer sets. The closed set, PRDB, holds 106,413 human NeurIPS and ICLR reviews from 2016 to 2024, plus paired model reviews.
| Method | dev F1 | test F1 | RAID TPR at 5% FPR |
| Fine-tuned gemma-2-2b | 98.0 | 75.2 | 74.9 |
| gram2vec | 77.9 | 72.0 | 43.6 |
| Binoculars | 73.1 | 81.4 | 79.1 |
| BASE-SAE, τ>0.7 | 89.4 | 80.9 | 78.6 |
| FT-SAE, τ>0.7 | 91.0 | 92.2 | 84.4 |
| FT-MODEL+SAE, τ>0.7 | 96.8 | 75.4 | 64.5 |
In-distribution, FT-MODEL+SAE stays with the black box: dev F1 97.5 with no mask against 98.0 for fine-tuned Gemma, and 96.8 at τ>0.7. Off the training split it falls with the black box. Test F1 is 75.4, next to 75.2.
The features that travel sit in the base model. FT-SAE at τ>0.7 scores 92.2 F1 on the COLING test set, 17 points above fine-tuned Gemma, and above gram2vec at 72.0, Binoculars at 81.4, and ReMoDetect at 86.7. The same 92.2 holds with the mask off. BASE-SAE is more sensitive to the mask: test F1 falls from 88.9 to 80.9, about even with Binoculars. Unmasked, its RAID true-positive rate at 5% FPR is 86.4, against 74.9 for fine-tuned Gemma and 43.6 for gram2vec. On that same test split, RoBERTa scores 74.9 F1 and fine-tuned Gemma scores 75.2.
On PRDB, every trained classifier, gram2vec included, is near perfect even at a very low false-positive rate. Shortcuts, not separation, are what the audit finds. For FT-MODEL+SAE at τ>0.85, the feature described as double colons after bold markdown headers appears in 11 human reviews and 54,293 GPT-4o reviews. Zeroing it moves the logit by +0.672 on average. On the Llama side, an opening square bracket in lists is 7 human reviews against 22,541. Human-leaning features include sentence-initial "I" (78,205 human vs 10,209 AI, logit shift -0.919), "seem", and "see" used as a cross-reference. On COLING, a strong FT-SAE feature fires on stock refusals such as "As an AI language model". The top BASE-SAE features cluster on HTML attributes and license boilerplate.
A feature-level paraphrase checks whether the description actually drives the score. Gemini rewrites a document to remove only the named feature. Importance and the logit change correlate at Spearman 0.69 to 0.81 on PRDB, with sign agreement of 89% to 99%, and only 0.36 to 0.62 on COLING. Above τ of 0.7, FT-MODEL+SAE moves from 0.54 to 0.62 on COLING and from 0.69 to 0.80 on PRDB. A document-level attack rewrites 2,500 AI texts and asks for edits under 30% of tokens. With no feature list, at most 8.7% of decisions flip, often none. Putting the active features in the prompt and iterating three times is reported in the main text at a 60% to 100% flip rate, with under 40% of tokens changed. The appendix records an exception: FT-SAE on PRDB reaches 21.8% attack success after three passes, because the evidence is spread over about 150 densely firing features and a list of ten cannot cover them.
In review, teaching, and compliance, a decision you can spread out on the table is worth more than a fraction of in-distribution F1. An auditor sees "double colon, +0.672" rather than a bare probability. Formatting features can be kept or dropped one at a time. Kantamneni et al. argue for deleting formatting cues wholesale. A punctuation habit that shows up across generators is a fair tell.
The result that should change a roadmap is the base model. No detection fine-tune, only a logistic regression, and transfer beats a detector trained for the task. The trainable parameters are the linear weights. Pretraining already contains linearly separable directions for AI text. Better reconstruction does not rank those directions. On COLING layer 7, fine-tuning the SAE cuts unexplained variance from 11.27% to 3.71%, and out-of-distribution detection gets worse.
Handing over the feature list also hands over the edit list. Auditability and easier evasion are the same interface.
Auto-interp descriptions run high recall and low precision. When a feature appears only inside one local pattern, the description drops the condition and can still score as a perfect simulation. An informal check that turned descriptions into regular expressions found about half the features at 99.5 F1 or better. The rest showed the recall bias to varying degrees. gemini-3.1-flash-lite assigns the score, and the paraphrase attacks use that same model. There is no large human rating.
The only base model is gemma-2-2b with GemmaScope. There is no ablation that removes HTML, license boilerplate, and refusal templates and reruns the transfer numbers. The appendix notes that arXiv, Wikipedia, and QA text recur across COLING splits, so the official out-of-distribution cut is not a clean domain shift.
A saturated PRDB does not show that AI peer review is solved. Preprocessing wraps section titles in bold markdown. The preprocessing note says a shared template should weaken format features. The highest-ranked tells are still double colons and square brackets. Those counts do not separate a model tic from a dataset artifact.