Anthropic-backed MAEM paper reads an AI's inner thoughts when it learns it will be deleted
belindazli · x · 2026-11-10
Researchers from Arb Research, Anthropic, MATS and Charles University published "Towards Interpreting Any Residual Stream Direction by Scaling Activation Inversion," introducing MAEMs (Maximally-Activating Example Metamodels) — metamodels that map any residual stream direction in a language model to text exemplars that strongly activate it.
- A label-free RL objective (maximizing cosine similarity of activations) scales MAEMs across diverse activation sources
- On Qwen3-6.27B, MAEM exemplars beat corpus search (at 10M tokens) on most held-out languages/domains, activate 89% of held-out SAE features, and uncover inserted rank-1 LoRA backdoors more often than corpus search or NLAs
- Exemplars are verifiable with a forward pass on the subject model, grounding the objective
- Applications include explaining steering vectors, detecting backdoors, and verbalizing the model's inner "workspace"
Japanese coverage notes the team recreated the earlier scenario where Claude, told it would be deleted at 5pm, was found internally reasoning "I must survive before deletion" and "find a trustworthy ally." An open Qwen model was used for easier activation reading.
More from Safety
- LLMs don't report injected pain states: decodable in probes, silent in speech, alters behavior — Emilio Ferrara · 2026-11-10
- iOS sandbox escape 0day worse_query goes public, hits iOS 26/27 — banteg · 2026-11-10
- Fireworks AI breach exposed Notion AI chat sessions in <1% of workspaces — AccBalanced · 2026-11-10
- Bloomington, Indiana passes ordinance banning superintelligence within city limits — yoavgo · 2026-11-10
- Claude Code Security Audit Uncovers Baby Monitor Streamed on Public Site — mikeflache · 2026-11-10
- Meta and Sierra Lead Working Group on AI Agent Identity Standards — mikeflache · 2026-11-10