Interp researcher pushes back: probes have advanced well beyond pre-LLM-era techniques

aryaman2020 · x · 2026-09-07

In a debate over the value of probes, the author rebuts the claim that probes are simple and unimproved by mechanistic interpretability research:

Frontier labs deploying probes in production largely rely on interpretability researchers, who should claim probes as a big interp win
Cites two academic examples where causal interp advances improved probes and vice versa
Probe architectures (attention probes, whole transformer probes) and hyperparameter tuning are now very different from the pre-LLM era
Asks: if not the post-LLM wave of interp research, who do you credit—and who do you fund to work on this?

More from AGI Musings