Dead Salmon fMRI lesson comes for AI interpretability in new arXiv paper
sebkrier · x · 2026-10-16
A new arXiv paper, The Dead Salmons of AI Interpretability (2512.18792, Université Grenoble Alpes et al.), revisits the famous 2009 fMRI study where a dead Atlantic salmon showed "brain activity" in response to social-emotion images — a classic case of uncorrected statistical noise.
The authors argue interpretability research is riddled with analogous artifacts: feature attribution, probing, sparse autoencoders, and even causal analyses can produce plausible-looking explanations for randomly initialized networks. They propose a statistical-causal reframing — treating explanations as parameters inferred from computational traces, methods as statistical estimators, and findings as hypotheses to be tested against explicit alternatives with quantified uncertainty. They also flag identifiability problems in common interpretability queries as a root cause of false discoveries, poor generalizability, and high variance.
More from Research
- DIY benchmark ranks 18 models on subjective judgments, GLM-5.3 tops consensus match — arkuto · 2026-10-16
- UCSD Researchers Propose Dataset Watermarking Detectable via Black-Box API Access — yuxiangw_cs · 2026-10-16
- EmbodiedEvo tops RoboDojo as agents inherit reusable skills and lessons — Aiden_Tech_Ai · 2026-10-16
- CHI reviewer laments: papers' Discussion sections now cite nothing — IanArawjo · 2026-10-16
- Casper: Understanding 90% of neurons ≠ understanding the network — StephenLCasper · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16