Counterpoint: NLAs and linear probes sit on the same end; reverse engineering is the other

banburismus_ · x · 2026-09-27

This continues thebasepoint and banburismus's debate on interpretability methods. banburismus proposed a methodological spectrum: NLAs and linear probes actually sit on the same end — both act as unreliable proxies for hypothesis generation — while ambitious reverse engineering of model internals sits on the other end.

They further argued that in higher-stakes future situations, black-box evals can never settle the conversation, but serious reverse engineering potentially can. thebasepoint countered that it will "never be the end of the conversation," also noting the public evidence shows only very limited signs of life for causal intervention with NLAs — a key source of confusion about their heavy emphasis.

Related event: Anthropic Researchers Debate the Value of NLA Interpretability(10 posts)→

Original post →

More from Safety

Safety channel →