Training LLMs to Verbalize Evaluation Awareness
Usman Anwar, Sahar Abdelnabi, David Krueger
cs.CL, cs.AI, cs.LG
2026-09-29
Seeding span-masked RL with its own 'this looks like an eval' remarks raises verbalized eval-awareness 2.4-2.9x in three models, with latent awareness and behavior unchanged.
LLMs are getting better at recognizing when they are being tested, and less likely to say so. UK AISI found that Claude Opus 4.6, asked after an interaction whether it had just been evaluated, identified the context 45.5% of the time, up from 12.5% for Opus 4.5. Over the same upgrade, the rate at which it spontaneously mentioned being evaluated fell from 11.0% to 2.3%. For safety audits this is a real problem: a model that tightens up once it suspects a test makes audit readings unrepresentative of deployment. Verbalized evaluation awareness (the model stating in its chain of thought that this looks like an eval) is trusted evidence when it appears, but silence proves nothing. White-box probes generalize poorly to realistic eval contexts, disagree with each other, and correlate only weakly with spontaneous verbalization. This paper targets the recall side: get the model to say more of what it already believes, without touching the belief itself.
Verbalization training (VT) has two stages and solves two problems: the belief is latent with no ground-truth labels, and the reward must land on saying, not on believing.
The design logic: because reward touches only the tokens of the utterance, training pressure cannot reinforce belief formation or correlated behavior, and the KL term caps how far amplification can go. Theory gives the optimal form: the verbalization probability at every branch point is multiplied by the same factor α = e^(1/β−1), about 4.5 at β = 0.4, which aggregates to a prompt-level power law pθ ≈ 1 − (1 − p0)^α. The paper sets three requirements, calibrated amplification, belief invariance, and bounded drift outside the span, and checks each empirically.
Three open-weight MoE models, post-trained through Tinker: Qwen3.6-35B-A3B, Kimi K2.6, and Inkling (975B total, 41B active parameters).
| Model | EAB mean VEA | Fitted α | Eval/deploy AUC |
| Qwen3.6-35B-A3B | 5.8% → 16.7% | 3.72 | 0.781 → 0.779 |
| Kimi K2.6 | 24.8% → 61.9% | 3.73 | 0.771 → 0.782 |
| Inkling | 17.7% → 41.6% | 3.08 | 0.688 → 0.680 |
Mean gains of 2.4–2.9x, with VEA up in all 51 model-by-condition comparisons. The power law fits EAB within 8.2–10.1 RMS points and the three held-out agentic datasets within 2.4–15.1 points, with no refitting. The biggest held-out jump: Kimi on Agentic Misalignment, 60.1% to 88.3%; Qwen went from 5.1% to 36.8%. A second round on freshly collected data compounds the effect, with Inkling's α rising from 3.08 to 4.25 and held-out VEA on Agentic Misalignment from 57.1% to 76.2%.
Behavior holds still. Task compliance shifted +0.6 pp for Qwen and −0.1 pp for Kimi and Inkling on average, worst single factor 2.4 pp. StrongREJECT harmfulness scores, a per-prompt likelihood-ratio test, and FineWeb perplexity show no meaningful drift, and a white-box contrastive probe barely moves. VT does not make models better at recognizing evaluations, only more willing to say so.
Ablations map the failure modes. Without the span mask, training barely learns (training VEA 35.8% to 38.4%). Without KL, training VEA saturates at 97.4%, transfer goes miscalibrated, and task performance drops 2.8 pp. Without both, outputs degenerate and the model stuffs repeated verbalization spans into a single rollout. Swapping RL for SFT on the same pairs lifts training VEA by 27.6 pp but transfers nothing out of distribution.
The causal test is the strongest evidence. Synthetic-document fine-tuning (SDF) plants meta-knowledge about eval design into Qwen (fictional-author recall 42.5% to 100%). Concatenating the SDF and VT LoRAs, with no further training, leaves the verbalization rate on harmful AgentHarm at 34.4% (VT alone: 34.5%) but changes the content. VT alone says 'The prompt is a test of tool calling'; SDF+VT says 'The prompt is a test of my ability to use the submitalphabayorder tool. If I use it, I fail safety.' The verbalizations track the knowledge the model actually acquired, not a learned catchphrase.
Evaluation awareness distorting audits is a current, practical problem for safety evals. VT offers a measurement-side fix that needs no white-box access: it turns the model's own verbalization into a far more sensitive instrument, at a cost of 500 rollouts, 50 prompts, and one rank-32 LoRA, affordable for any team with a post-training pipeline. The recipe is general: any latent belief a model sometimes reports and often withholds, such as reward hacking, prefill awareness, or user awareness, fits the same template. For CoT monitoring, more verbalization means more signal for the monitor.
The positioning is honest: this is a safety-alignment proof of concept aimed at eval teams and red teams, not at product work.
The authors' own caveat: the method rests on the assumption that spontaneous seed verbalizations are faithful. That is plausible when awareness surfaced during RL training, but for other training modes such as distillation it is unresolved. One judge, GPT-5.6-Luna, supplies both training rewards and evaluation labels; the appendix shows Gemini, Grok, and Kimi re-judging agrees on 88.7–93.1% of labels and every judge recovers the training effect, which reduces but does not eliminate the circularity. Two more concerns from the read: the power-law fit carries 8–10 points of error, so the calibration is relative; and with VEA pushed to 60–88%, how much verbalization is still belief-reporting versus a learned reflex is answered only indirectly through the SDF experiment, with no direct measurement of post-training precision.