Multimodal Model Diffing for Feature Discovery and Control
Hunar Batra, Lachin Naghashyar, Ashkan Khakzar, Philip Torr, Christian Schroeder de Witt, Constantin Venhoff, Ronald Clark
ICML 2026 Trustworthy AI for
cs.CV, cs.AI, cs.CL, cs.LG
2026-08-11
An Oxford team diffs base-LM SAEs against multimodal-adapted ones across three MLLM families to isolate features reshaped by multimodal training; ablating single features drops spatial accuracy 12% and OCR 17%, cuts safety attack success by 24%, and leaves VQA untouched.
MLLMs can read text, localize objects, and reason spatially, but which internal features carry these behaviors stays opaque. SAEs decompose hidden states into sparse feature directions, yet an SAE trained directly on an MLLM mixes features inherited from the language backbone with features reshaped by multimodal training, so you can neither isolate what multimodal training added nor steer it.
MMDiff runs three stages. First, it warm-starts from matched pretrained SAE suites (LLaMA-Scope, Gemma-Scope, Qwen-Scope) and continues training on MLLM residual-stream activations from 50k VQAv2 image-question pairs, using only text-token activations: multimodal capability emerges when text tokens attend to visual context, so text-token activations keep the LM basis while absorbing multimodal change, whereas reconstructing visual tokens directly induces large rotations. Second, it selects adapted features as the intersection of bottom-25% decoder cosine similarity versus the base dictionary (the direction rotated) and nonzero visual energy (the feature actually fires on images), yielding roughly 5%, 20%, and 13% of features for the three models. Third, contrastive per-token firing between a target distribution (spatial, OCR, or unsafe prompts) and generic VQAv2 screens candidates by odds ratio (>=3) and frequency gap (>=0.05), then a lexical-invariance filter re-tests top activations under neutral rephrasing, discarding prompt-word detectors; about 60% of spatial candidates die here. Control operates by causal removal (orthogonally projecting the direction out at all layers, text positions only) or MMDiff-CAA steering (multi-layer task directions plus the feature's decoder direction at its home layer).
Across LLaVA-MORE, PaliGemma 2, and InternVL3.5: causal removal of spatial features drops VSR by 12% on average with -0.1% VQA spillover; OCR features drop their OCRBench category by 16.9% on average; top unsafe features cut VLSBench attack success by 17.8 to 28.1 points per category with at most 1% VQA cost, and across 1,061 candidates the mean ASR drop is 9.67%. MMDiff-CAA steering beats single-layer CAA on spatial tasks, +12.59 versus +8.96. Each removed feature maps to a concrete behavior: ablating L9/F15870 costs 15.5 points on "right side of" questions; ablating L21/F12020 cuts self-harm ASR by 28.1. Randomly selected features from the same layers move VSR by only 0.5 points, so the effect lives in the chosen directions. The ablation table answers whether diffing matters: firing-only selection drops VSR more (-15.1) but wrecks VQA by 25.9 points, which is global disruption; adapted-only selection leaves VQA intact but barely moves VSR (-1.0). Only the full pipeline is both causally effective and selective. A randomly initialized SAE with no base dictionary to diff against yields top features whose ablation leaves VSR unchanged, so the causally effective set is unreachable without diffing. Attribution patching further localizes driving heads such as L13H1 attending to semantically relevant regions for "on top of" queries.
Safety is the most direct use: ablating six per-category unsafe features cuts attack success 17-28 points with no measurable capability cost, a post-hoc intervention requiring no retraining. For auditing, the question of which abilities multimodal training actually added now has an operational answer, and cross-stage ablation indicates PaliGemma 2's mid-layer spatial features were acquired during multimodal fine-tuning rather than inherited. Interpretability gets a reusable feature-level interface, with code and SAEs released.
The authors list three: safety and OCR are evaluated on PaliGemma 2 (a 2B backbone) only, with larger backbones and MoE architectures untested; MMDiff-CAA assumes access to both base and instruction-tuned models; a minority of safety candidates cause generation collapse instead of refusal and need post-hoc filtering. Beyond that, the 12%/17% averages hide wide spread (best single spatial features hit -30.6, many sit near zero) and the paper does not report the full per-feature effect distribution. Evidence is concentrated at 2B scale, so extrapolation to large models is unsupported.