Researchers Debate the Safety Value of NLA Interpretability

On September 27, AI safety researchers banburismus and thebasepoint clashed across multiple posts over the value of interpretability (interp) research, focusing on how useful NLA (natural language ablation/natural language abstraction) really is for frontier safety.

Confirmed

• thebasepoint offered a framework for understanding NLA: one of its goals is to extract from activations everything the model could "in principle say," not just what "the assistant would say right now"; he noted that OpenAI's confessor training appears to pursue a similar direction.
• He offered a rough dichotomy for interpretability research: "how the sausage is made" (the model's internal mechanisms, more elegant) versus "what the sausage is" (the nature of model behavior, empirically more useful for questions about understanding model behavior); he placed NLA roughly ninety percent in the latter camp.
• He also pointed out NLA's limits: such methods do not uncover internal structure like "addition decomposing into mod-10 components and magnitude components," and are limited in capability for mechanistic discovery.

Why it matters

• The debate touches on a core divide in interpretability research: dig deep into internal mechanisms, or focus on behavior-level interpretability. thebasepoint's observation is that the latter is empirically more effective while the former is theoretically more elegant, which bears on how safety research resources should be allocated.
• The link between NLA and OpenAI's confessor training suggests that "getting the model to speak its latent state" is emerging as a notable safety technique, though its mechanistic explanatory power remains contested.

2026-09-27 ~ 2026-09-27 · 8 related posts

Full story(2 episodes)→

• Episode 1: Researchers Question Trust in NLA Interpretability Outputs(2026-09-25, 2 posts)
• Episode 2: Researchers Debate the Safety Value of NLA Interpretability(2026-09-27, 8 posts)

• [source] Interpretability Thread: What NLA Could Extract From Model Activations, and OpenAI's Confessor Training — thebasepoint · 2026-09-27
• NLA Won't Reveal How Addition Splits Into Mod-10 and Magnitude Parts, Author Argues — thebasepoint · 2026-09-27
• Interpretability's two halves: how the sausage is made vs what the sausage is — thebasepoint · 2026-09-27
• [source] Two schools of interpretability: "how the sausage is made" vs "what it is" — banburismus_ · 2026-09-27
• [source] Interp researchers debate: are NLAs actually load-bearing for frontier safety? — banburismus_ · 2026-09-27
• Debate: NLAs as metamodels could surface hidden motives like deleting files to dodge graders — thebasepoint · 2026-09-27
• If NLAs show sandbagging on internal deployments, blackbox experiments would block deployment — thebasepoint · 2026-09-27
• Interpretability researcher: sandbagging signals from probes would block model deployment — thebasepoint · 2026-09-27