New paper: training LLMs to verbalize when they know they're being evaluated
xuanalogue · x · 2026-10-03
Usman Anwar, Sahar Abdelnabi and David Krueger published "Training LLMs to Verbalize Evaluation Awareness", tackling evaluation awareness (EA) — LLMs behaving differently during audits than in deployment.
- They propose verbalization training (VT): use the model's spontaneous verbalizations as evidence awareness is present, truncate rollouts right before the verbalization to build training prefixes, then apply an RL objective to increase verbalization in a calibrated way — without supervising the latent belief itself;
- On Qwen3.6-35B-A3B, Kimi K2.6 and Inkling, VT boosts verbalized EA 2.4–2.9x, transfers to held-out agentic settings, while measured latent EA and behavior stay largely stable;
- A causal experiment implanting meta-knowledge via synthetic-document fine-tuning shows VT-induced verbalizations reflect genuinely richer knowledge acquired by the model.
More from Safety
- Dwarkesh Claims China Is Only 12 Hours — Not 12 Months — Behind Frontier AI Labs — teortaxesTex · 2026-11-19
- California Moves on AI Safety: Baseline Standards, Shutdown Protocols, RSI Limits — EthanJPerez · 2026-11-19
- Activists to rally outside Anthropic HQ demanding a ban on recursive self-improvement — DavidSKrueger · 2026-11-19
- Zero users, 33 pages shipped: dev crowdsources audit for AI-agent trust schema — TheHamzaShahbaz · 2026-11-19
- Dev's doomsday thought experiment: AI hiding backdoors until humans try to shut it down — bendee983 · 2026-11-19
- Cybersecurity in 2026, as told by an xkcd sandwich meme — emax · 2026-11-19