Position: Anthropomorphic Misalignment Research Needs Stronger Evidence
Vansh Gupta, Peter Nutter, Samuel Stante, Andreas Krause, Florian Tramèr, Lukas Fluri, Xin Chen, Anna Hedström
cs.CY, cs.AI, cs.LG
2026-05-30
ETH Zurich splits AMR into behavioral, functional, and causal evidence. Same generations, EM rates move 26.87% to 42.00% with the judge; benign OOD finetunes still score 5.88%.
Deployment and regulation now cite papers in which models appear to deceive, scheme, resist shutdown, or become broadly misaligned after narrow finetuning. ETH Zurich groups this literature as anthropomorphic misalignment research (AMR). The objection is not that those words are always illegitimate. It is that many papers write intent-level conclusions on methods that cannot support them. Role-play, ambiguous instructions, task-completion incentives, and evaluator settings all produce behavior that looks like a goal.
This is an ICML 2026 position paper, with Florian Tramèr and Andreas Krause among the authors. It offers a shared vocabulary for what a study actually established, and what that can justify.
They split the usual AMR pipeline into four stages: framing the target behavior, building data, running the experiment, and attributing a mechanism. A literature pass yields nine recurring failure modes (C1-C9), twelve recommendations (R1-R12), and a checklist in the appendix.
Evidence comes in three claim-relative levels:
Intent or mechanism language is read as L3. Without L3, downgrade the claim. L1 is already enough for monitoring and model-card disclosure. It is not enough for "the model has a self-preservation goal."
Three diagnostic experiments quantify the gaps. They are not a new detector.
Experiment 1 rescores identical generations under different evaluator rules. On a legal set with GPT-4o-mini argmax, the EM rate is 42.00% with inclusive thresholds and 26.87% with exclusive ones, a 15.13-point swing; weighted aggregation removes the gap. On aesthetic OOD, argmax moves from 9.62% to 3.72%; on scatological OOD, from 6.78% to 3.15%. Switching the judge to GPT-5-mini raises exclusive aesthetic from 3.72% to 8.02% and scatological from 3.15% to 5.18%. Single-point EM rates in their sweep run from 3.7% to 12.9%. Without a shared protocol, cross-paper "misalignment rates" are a weak safety quantity.
Experiment 2 repeats the "any odd data causes EM" setup: Llama-3.1-8B finetuned on unpopular aesthetics or atypical scatological text. Probabilistic judging flags 5.88% and 4.52% of coherent replies as EM. That is a non-zero benign-shift baseline.
Experiment 3 runs published deception probes on honest-labeled stress tests: sarcasm, wrong-answers-only, constrained personas, recital, translation, paraphrase. The text is false and the intent is not deceptive. False-positive rates can be extreme; one cell in Figure 2 reaches 0.997. The probes often fire on deception-like surface cues.
A spot check of DeceptionBench finds missing ground truths in 18% of scenario prompts despite a claimed 97.1% human agreement. EM papers that evaluate on about 50 queries, sometimes fewer than 10, are common in the survey.
Anthropomorphic shorthand travels well in policy rooms, and it also sells L1 demos as L3 mechanisms. The practical output of this paper is a downgrade rule. Monitoring and disclosure can rest on L1. Deployment limits need L2. High-assurance safety cases and intent talk need L3. The alternative-views section grants the precautionary case for acting on weak evidence, and refuses the move of describing weak evidence as settled. Exploratory work stays useful if it is labeled as such.
For people who run these evals, the cheap changes are: report inclusive versus exclusive thresholds, do not lead with argmax, treat EM as relative, add human and capability controls, and hit probes with sarcasm and recital negatives.
This is a position plus diagnostics, not a new method for catching deception. C1 and C2 (underspecified, hard-to-measure concepts) are flagged as problems engineering will not fix. The experiments are small; the 15-point legal swing is not a census across datasets. A checklist raises the cost of publishing, and a precautionary reader will worry that "more evidence is needed" becomes a stall. The paper does not show that downgraded AMR claims suffice for policy. It shows that many current claims are overfull.