Models can detect concept injections in latent space: detection hits 39.9% with prompting
voooooogel · x · 2026-11-08
The arXiv paper "Latent Introspection: Models Can Detect Prior Concept Injections" (submitted to ICML 2026) uncovers a latent introspective capacity in a Qwen 32B model: it can detect when concepts were injected into its earlier context and identify which one — even while denying it in sampled outputs.
Key findings
- Logit lens analysis: clear detection signals in the residual stream, attenuated in final layers, hence invisible at the output level
- Dramatically steerable: prompting the model with accurate information about AI introspection mechanisms raises injection sensitivity from 0.3% to 39.9% with only a 0.6% rise in false positives; mutual information between nine injected/recovered concepts rises from 0.61 to 1.05 bits, ruling out generic noise
Implications: models possess a surprising, easily overlooked capacity for introspection and steering awareness, relevant to latent reasoning and AI safety. Co-author vooooogel notes recent NeurIPS-era updates complicate the picture somewhat.
More from Safety
- MCP tool descriptions silently change — and none of the 4 major AI coding tools warn you — abhiramreddykalluru · 2026-11-08
- Vercel CLI caught uploading all your code by default with AI training opt-in enabled — kingslyj · 2026-11-08
- XyorNinja CEO: No playbook for stopping 1,000 AI agents attacking a network at once — zacharynado · 2026-11-08
- tszzl Calls for Total Chip and AI Export Ban on China, Allied Compliance Required — tszzl · 2026-11-08
- Researchers Slam Waymo Blockades: AVs Crash 68% Less Yet Face Drunk-Driving-Style Policy Paradox — QuintinPope5 · 2026-11-08
- AI law firm Crosby secured $5M liability insurance for its AI agents — graceisford · 2026-11-08