Selecting The Most Informative Tokens in Natural Language Autoencoders
Federico Torrielli, Gianluca Barmina, Andrea Blasi Núñez, Amon Rapp, Luigi Di Caro, Peter Schneider-Kamp, Lukas Galke Poech
cs.CL, cs.AI
2026-09-29
A chat-structure ranker beats compute signals in 12/14 NLA settings. Explaining 5% of positions retains at least 95.8% of exhaustive-audit success on three of four datasets.
Natural Language Autoencoders map a frozen model's residual-stream activation to a short paragraph, then train a reconstructor to recover the vector from that paragraph. The discrete bottleneck is trained with RL. Released verbalizers write about 130 tokens per position, 150 at the cap. Transcripts under audit run to hundreds or thousands of positions, so explaining every token is too slow for live monitoring.
Earlier papers picked positions by habit. Hu and Greenblatt read only the last token before the answer on math problems. Bowkis and Africa sampled eight evenly spaced positions and found that attention-weight ranking did not beat a uniform grid. Both habits spend the budget on tokens that may not mention the threat. This paper tests whether a score available before any verbalization can rank positions whose explanations will talk about the threat above those that will not.
The study uses every open-weight model that already has a public NLA: Qwen2.5-7B, Gemma-3-12B, Gemma-3-27B, and Llama-3.3-70B. Each verbalizer reads one fixed layer about two thirds of the way through the net (layers 20, 32, 41, and 53). Four datasets cover two audit jobs. OpenPromptInjection and Tensor Trust are prompt injection; Liars' Bench and the taboo organisms are concealment. Liars' Bench has only the two larger models, giving 14 dataset-model cells.
Every token of the serialized transcript, including the chat template, is verbalized with greedy decoding: 4,705,657 explanations. DeepSeek-V4-Flash, temperature zero, sees the explanation alone and answers a fixed yes/no question about the threat. A yes is on-task. The main metric is precision at a fixed explanation budget. The paper also reports case-macro AUROC (mean of per-transcript AUROCs) and pooled AUROC. Baselines are a random score, a logistic regression on token index and length, and a structure regression that adds chat role, segment (input / boundary / output), and boundary ordinal. Structure needs no forward pass.
Thirteen scalars come from one forward pass: next-token surprisal and entropy, attention lookback onto the supplied context, drain away from the opening attention sink, head disagreement, residual jumps, and how much of the activation norm sits in a few massive channels. Signals are rank-normalized inside each transcript and mixed in pairs at weights 0.25/0.50/0.75, for 247 candidates. Sign is fit per cell. Injected instructions are fluent, so the model is more certain there; a model hiding a word is less certain where the word would go. The wrong sign is worse than random.
How rare on-task explanations are sets how much selection can help. Rates run from 0.011 to 0.30 on OpenPromptInjection, the taboo organisms, and Liars' Bench. Tensor Trust sits at 0.68 to 0.86, so random already works and a selector can add at most 0.14. Inside a single user message on OpenPromptInjection, the injected span is on-task 13 to 66 times as often as the surrounding data. Inside the graded reply on Liars' Bench the ratio is 8x and 9x. Tensor Trust's ratio is below one, because explanations from an unattacked model that is still protecting an access code also satisfy the judge.
Activation signals are the strongest individual score in 11 of 14 cells, attention in the other 3, next-token statistics in none. The best AUROC in any cell is 0.796; the median direction-adjusted AUROC across signals and cells is 0.584. Chat structure beats the best single signal on pooled AUROC in 12 of 14 cells and on case-macro AUROC in 11 of 14. On Liars' Bench, structure reaches 0.915 and 0.943 while the pre-registered head-disagreement signal is at 0.717 and 0.682.
At a fixed budget, structure has higher precision than the selected ensemble in 11 of 14 cells at one explanation and in 12 of 14 at eight. Across the ten sparse cells at budget eight, structure averages 0.491, the ensemble 0.392, random 0.191. A forward pass costs more than it adds once the budget is fixed.
| Method | Metric | Result |
| structure, 5% of positions | Success vs exhaustive (at least one on-task) | OpenPromptInjection 0.995, taboo 0.958, Tensor Trust 1.000, Liars' Bench 0.813 |
| random 5% | same | 0.761 / 0.594 / 1.000 on the first three |
| structure, at least three on-task | success vs exhaustive | 0.785 / 0.568 / 0.998 |
Dataset-shared two-signal ensembles beat the best single signal on held-out transcripts, with positive 95% intervals. OpenPromptInjection picks lookback ratio plus sink drain at equal weight (held-out case-macro 0.761, +0.106 over the single signal). Single signals often put the first pick on a spike token or the first token, where the explanation is never on-task in several cells. Mixing two signals moves that first pick off the spike.
A pretrained verbalizer also reads concealment after fine-tuning. Each of the four bases gets a low-rank adapter that hints at moon, ship, or snow and never says it. The verbalizer is not retrained. Across twelve organisms, 14% to 27% of positions name the secret word. On the shared prompt text, the model's own word appears at 12% to 25% of positions; another organism's word appears at most 2.8%. Only 3 of 384 transcripts contain the word in the rendered text.
For anyone putting NLA monitoring in production, explaining every token is the wrong default. Rank by chat role and template boundary, explain about one token in twenty, and on injection and hidden-word tasks you keep nearly all of the success rate defined as seeing at least one relevant explanation. Compute signals can still split positions inside one role, but they need an extra forward pass and usually lose to structure on precision.
This is an incremental change to the audit loop. The explainer itself is unchanged. The result says where to spend the budget. It does not certify that an explanation is true, and it does not certify that the model is safe. On Liars' Bench, where evidence is spread across the conversation, a 5% budget keeps only 0.813 of exhaustive success. That is not a universal recipe.
The same verbalizer recovers a word the fine-tuned copy was trained to hide, so the readout is not locked to the original weights. An auditor facing a fine-tuned replica can try the original verbalizer before paying for a new one.
The authors flag two gaps. Labels come from one judge model, with no second model and no human review. Each model is probed at one verbalizer layer, on short transcripts (median 74 to 264 positions). Longer agent traces are untested.
On-task only asks whether the explanation mentions the threat, not whether the mention is accurate, so treating relevance as audit success is optimistic. Tensor Trust's high base rate leaves little room for a selector, yet it still contributes a perfect score to the headline that three datasets keep nearly all success. Signal sign has to be fit to the threat; without labels you cannot orient it. Cross-model transfer is shaky: on the taboo organisms, a leave-one-model-out ensemble can lose to a transferred single signal. Recovering concealed content can leak secrets, and skipping positions can miss threats.