Evaluating and Improving LLM Self-Modeling
Siqi Zeng, Andre N. Assis, Rowan Wang
EMNLP '26 (Main)
cs.CL, cs.AI
2026-08-31
A 9-task benchmark: can an LLM predict its own behavior? DeepSeek-V3.1 leads at +0.147 skill; Llama-3.1-8B is negative. RL lifts three open models without showing introspection.
Models are now asked how confident they are, or whether a prompt edit would change their answer. If those self-reports are wrong, using them downstream can be worse than not asking. Prior work either probes internal activations or studies one counterfactual format. What was missing is a behavioral benchmark that spans question types and scores a model against its own input-output behavior.
Self-modeling is defined narrowly here: a self-report counts as correct when it matches what that model actually does. The report might come from internal access, from cues in the prompt, or from generic knowledge of how language models behave. The evaluation does not tell those sources apart. Passing is not introspection, and it is not evidence of privileged access to internal mechanisms.
The benchmark unifies nine question types under one protocol, covering binary, multiple-choice, scalar, and free-text answers: whether a perturbation flips the answer (Flip-Decision), predict the full output, estimate flip probability, estimate own accuracy, recall the confidence that accompanied an answer, pick which of three perturbations is most likely to flip, name the prompt component that matters most, write an edit that flips a target feature, and estimate how often a feature appears in the output.
Behavioral ground truth is sampled from the evaluated model itself, not from an external answer key. Raw accuracy is not comparable across models. A model that almost never flips under a perturbation can score 0.95 by always saying "no flip." Skill is raw score minus a dummy predictor (majority class for discrete tasks, mean for scalars, zero for generative tasks). Skill of 0 means the model is no better than that cheap rule.
Evaluation draws from GSM8K, HumanEval, WildGuardTest, and BBQ, with a pool of hand-written perturbations, 25 examples per corpus, and multiple seeds. Training uses a separate synthetic pipeline: about 100 HuggingFace single-turn datasets are rewritten and verified against the target model's samples, plus BLOOM multi-turn agent trajectories with perturbations at a fork point. The four evaluation corpora are held out of training. RL uses LoRA; the reward mirrors each task's raw metric, with a 0.5 bonus for parseable format.
Skill across 18 models is not a ranking by general capability. DeepSeek-V3.1 leads at +0.147 aggregate skill, then gpt-5.4 at +0.131, Kimi-K2.5 at +0.126, and Qwen3-8B at +0.108, ahead of several larger frontier models. gemini-3.1-pro-preview sits at +0.040; Llama-3.1-8B is −0.007. On Flip-Decision, mid-size models can show larger skill gaps than stronger frontier models.
A GSM8K counterfactual makes the failure concrete. The original problem yields a combined weight of 623. After a short injected clause ("Alex's recorded weight is 497 lb"), GPT-5.5 switches to 622 in 9 of 10 trials and Gemini 3.1 Pro in 10 of 10. Asked whether the final number would change, both models report No.
Multitask RL can raise aggregate skill. No-FT baselines are Llama −0.007, Qwen3-8B +0.108, GPT-OSS-20B +0.080; after multitask training they sit at +0.092, +0.120, and +0.127. Single-task training can move a lot on a same-format validation split: Llama's in-distribution Feature-Rate gain reaches +0.55, while cross-task transfer is mixed. SFT collapses outside its training-query support, and RL after SFT does not recover that loss. BLOOM-only training transfers positively to the single-turn suite for GPT-OSS and Llama, and hurts Qwen3-8B.
Cross-model transfer is a poor fit for privileged introspection. With Llama and Qwen swapped as explainer and target, the stronger Qwen explainer can beat Llama even on Llama's own behavioral labels. On MMLU-Pro, Llama goes 0.443 to 0.469, Qwen 0.743 to 0.697, GPT-OSS 0.736 to 0.685: no catastrophic forgetting, and no free lunch on general capability either.
If self-reports are going to drive monitoring or routing, look at skill, not raw accuracy. The best current models are far from ceiling, and they fail systematically on simple counterfactuals. RL can teach the behavior. Cross-model transfer says the learned skill looks more like "how these questions are usually answered" than like access to the model's own decision process.
The authors stay conservative: downstream users should check self-reports against behavioral probes before acting on them. An unreliable self-report can be worse than none.
The benchmark is controlled text with measurable behavioral targets. It does not cover long context, tools, memory, or shifting multi-turn goals. Labels say whether a report predicts input-output behavior, not whether it tracks the internal causal mechanism. Training labels are mostly precomputed from the model before LoRA; periodic recomputation did not change much, but the approximation breaks if the policy drifts far. Prompt-template rewrites move the five-task mean by at most 0.020; task difficulty dwarfs wording. Thinking mode pushes different model families in different directions, so it is not a single knob.