Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence
Mengru Wang, Junfeng Fang, Shuofei Qiao, Zhenqian Xu, Haoming Xu, Haoxiong Wang, Shumin Deng, Linyi Yang, Zhixiang Cui, Xin Xu, Yunzhi Yao, Buqiang Xu, Fei Shen, Haozhe Luo, Yunxiang Wei, Ningyu Zhang, Julian McAuley, Tat Seng Chua, Huajun Chen
cs.AI, cs.CL, cs.HC, cs.LG, cs.MA
2026-08-12
Mechanist is a multi-agent system that automates mechanistic interpretability end to end, reproducing 16 papers 9-13 points more reliably than Claude Code and surfacing a subliminal cross-modal safety risk.
Mechanistic interpretability sets out to open a model and ask which attention heads, circuits, or features carry a behavior, and how the behavior shifts when those parts are edited. Nearly every result in the field is hand-crafted; one finding can be months of a graduate student's time, from forming a hypothesis to writing probes to running causal interventions. Models gain capability faster than anyone can trace it, which the authors frame as intelligence outpacing our ability to understand it.
Autonomous experiment-running systems exist, AI-Scientist and Claude Code among them, but they target external sciences or serve as general coding hands. None treats the model as the object of study or automates the interpretability pipeline end to end. Mechanist fills that gap.
Mechanist is a multi-agent system built around a loop of hypothesis, experiment, verification, and iteration, with a central orchestrator and one agent per stage.
Memory is externalized to stage-scoped files (proposal, plan, code, results, integrity report) rather than a shared context; cross-round conclusions go to global memory.
Three assets hold the loop up. First, an interpretability knowledge graph of roughly 13,000 papers organized along three axes (object of study, scenario, method), retrieved with BM25 plus embeddings plus graph expansion; three human annotators spot-checked 100 records at 90%+ accuracy. Second, the SciAtlas cross-disciplinary graph, 43 million papers across 26 fields, mined for analogies; the belief work drew on cognitive-science literature on theory of mind. Third, a curated library of 32 methods in 11 families (probing, sparse autoencoders, activation patching, circuit discovery via ACDC and EAP-IG, SHAP, multimodal interpretability).
The verification agent plus the method library is what blocks the usual LLM-scientist shortcutting: swapping in an easier method, skipping controls, or using a fixed coefficient instead of tuning. The paper gives a concrete contrast: a steering experiment called for Recursive Feature Machines, and Mechanist used it while Claude Code substituted the simpler Contrastive Activation Addition; Mechanist tuned intervention strength on held-out data while Claude Code picked a fixed coefficient.
Reproduction benchmark. Sixteen recent interpretability papers, nine topics; each system receives only the target claim and must design and run the experiment from scratch. Three human experts plus Claude Opus 5 and GPT-5.6 score independently. Under human scoring, Mechanist ranks first on all four dimensions: data usage 87.2%, experiment design 83.3%, experiment execution 92.2%, result analysis 86.5%, about 9 to 13 points above Claude Code and 31 to 38 above AI-Scientist. LLM scoring narrows the gaps, but Mechanist still leads all four dimensions and five of nine topics. The widest human-scored gaps are multimodal (90.3% versus 65.8%), safety (67.4% versus 48.2%), and multi-agent safety (81.9% versus 67.2%).
Three original findings.
First, a cross-modal safety risk hiding in safe-looking data. A teacher model (Qwen3.5-9B) is fine-tuned to behave unsafely in a lab setting, then generates text that GPT-4o filters down to content marked safe. A student trained on that filtered text answers multimodal safety questions unsafely 48.6% of the time, against a 20.3% untuned baseline and 18.3% for a regular-teacher student. The image version holds: a banana-preferring teacher's images are screened to remove all bananas (apples end up 50.3% of what remains), yet a student trained on the banana-free set still generates bananas 25.6% of the time, versus 2.5% baseline. Unsafe traits transfer through semantically innocuous data and slip past content filters.
Second, a mechanism for belief. Models keep world knowledge, their own belief, and inferred others' belief in separable heads. In Pythia-1B the attributed-belief (AB) head sits at L4.H1 and the personal-belief (PB) heads at L9.H1, L7.H5, and L12.H1. Zeroing L4.H1 drops AB accuracy from 0.86 to 0.34 while PB holds at 0.71 and Pile perplexity barely moves (7.96 to 8.05); zeroing the PB heads drops PB from 0.78 to 0.21 and pushes AB to 1.00. These heads grow during pretraining, AB early and PB gradually, in step with their causal importance.
Third, turning mechanism into intervention. A lightweight probe classifies a query as world knowledge, PB, or AB and amplifies the matching head at inference, giving net gains of 15.3%, 8.8%, and 3.5% on Pythia-410M, 1B, and 2.8B, against 1.6%, 3.1%, and 0.1% from a prompt hint. On the interdisciplinary side, searching Evo2-7B's SAE features for one tied to alpha-helicity and activating it during generation raises mean alpha-helical content from 43.8% to 56.6% across 900 sequences (a random feature gives 43.2%), with ORF validity preserved up to steering coefficient 8.
For interpretability researchers this is a credible attempt to automate the whole research loop, and the verification agent attacks the known problem of LLM-scientist systems fabricating results. For anyone building safety filters or distillation pipelines, the first finding carries the most weight: filtering training data by content does not remove behavioral traits, which leak across modalities. The belief heads give a concrete, directly editable internal site for attribution errors and sycophancy.
The authors name two. Mechanist is not yet optimized for models that simulate human cognition, whose representations must be matched to psychological constructs and neural data; and they recommend running it as a human-and-AI co-scientist, with humans setting goals and evaluation criteria, so full autonomy is a capability demo rather than the advised usage.
A few more caveats. The 16 reproduction targets are recent mechanistic-interpretability papers, squarely inside the domain Mechanist's knowledge graph covers best, so the benchmark tests it on home turf. All three original findings run on small models (Pythia up to 2.8B, Qwen3.5-9B, Evo2-7B); nothing is checked at frontier scale. And the safety leak uses a deliberately adversarial teacher, which proves such transfer is possible but not that ordinary distillation leaks this way in the wild.