Paper questions interpretability: Black-box prompting often beats white-box tools
aryaman2020 · x · 2026-08-22
The paper "Pando" investigates if interpretability methods work when models won't explain themselves, introducing a benchmark to control for the "elicitation confounder."
Key Findings:
- Across 720 finetuned models, black-box prompting matches or exceeds white-box methods when explanations are faithful.
- When explanations are absent/misleading, gradient-based attribution improves accuracy by 3-5%, with Relevance Patching (RelP) offering the largest gains.
- Logit Lens, Sparse Autoencoders (SAE), and circuit tracing provided no reliable benefit.
- Gradients track decision computation, while other readouts are dominated by task representation.
More from Research
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- A Cookbook for Machine Learning: compiling ML goals into tractable optimization — goyal__pramod · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16