Interp researcher pushes back: probes have advanced well beyond pre-LLM-era techniques
aryaman2020 · x · 2026-09-07
In a debate over the value of probes, the author rebuts the claim that probes are simple and unimproved by mechanistic interpretability research:
- Frontier labs deploying probes in production largely rely on interpretability researchers, who should claim probes as a big interp win
- Cites two academic examples where causal interp advances improved probes and vice versa
- Probe architectures (attention probes, whole transformer probes) and hyperparameter tuning are now very different from the pre-LLM era
- Asks: if not the post-LLM wave of interp research, who do you credit—and who do you fund to work on this?
More from AGI Musings
- Dario's predicted timeline distilled: $1B one-person company in 2026, AI beating Nobel winners by 2027 — AccBalanced · 2026-10-24
- VC: X feels saturated but enterprise CEOs still aren't using AI — it's so early — karinanguyen · 2026-10-24
- AI founders argue the digital ecosystem needs rebuilding from scratch — creating jobs for now — max_paperclips · 2026-10-24
- New paper introduces "bottleneck pressure" to find where AI raises the value of work it can't do — soumitrashukla9 · 2026-10-24
- Meme/persuasion risk is more compelling than bio risk, argues teortaxesTex — teortaxesTex · 2026-10-24
- Why the 'AI creates 8x more jobs' claim doesn't hold: unpacking the U.S. Chamber survey — BuddyWeary653 · 2026-10-24