The NLA Credibility Debate: Interpretable AI Under Fire

Episode 1 · Researchers Question Trust in NLA Interpretability Outputs (2026-09-25, 2 posts)

Researchers are debating how much trust NLA-style mechanistic interpretability outputs deserve, with critics arguing current confidence resembles mere correlation-based readouts.

• Researcher questions NLA hype: how can we trust mechanistic interpretability outputs? — banburismus_ · 2026-09-25
• Are NLA outputs trusted like purely correlative readouts? An interpretability debate — thebasepoint · 2026-09-27

Episode 2 · Researchers Debate the Safety Value of NLA Interpretability (2026-09-27, 8 posts)

On September 27, AI safety researchers banburismus and thebasepoint clashed across multiple posts over the value of interpretability (interp) research, focusing on how useful NLA (natural language ablation/natural language abstraction) really is for frontier safety.

Confirmed

• thebasepoint offered a framework for understanding NLA: one of its goals is to extract from activations everything the model could "in principle say," not just what "the assistant would say right now"; he noted that OpenAI's confessor training appears to pursue a similar direction.
• He offered a rough dichotomy for interpretability research: "how the sausage is made" (the model's internal mechanisms, more elegant) versus "what the sausage is" (the nature of model behavior, empirically more useful for questions about understanding model behavior); he placed NLA roughly ninety percent in the latter camp.
• He also pointed out NLA's limits: such methods do not uncover internal structure like "addition decomposing into mod-10 components and magnitude components," and are limited in capability for mechanistic discovery.

Why it matters

• The debate touches on a core divide in interpretability research: dig deep into internal mechanisms, or focus on behavior-level interpretability. thebasepoint's observation is that the latter is empirically more effective while the former is theoretically more elegant, which bears on how safety research resources should be allocated.
• The link between NLA and OpenAI's confessor training suggests that "getting the model to speak its latent state" is emerging as a notable safety technique, though its mechanistic explanatory power remains contested.

• Interpretability Thread: What NLA Could Extract From Model Activations, and OpenAI's Confessor Training — thebasepoint · 2026-09-27
• NLA Won't Reveal How Addition Splits Into Mod-10 and Magnitude Parts, Author Argues — thebasepoint · 2026-09-27
• Interpretability's two halves: how the sausage is made vs what the sausage is — thebasepoint · 2026-09-27
• Two schools of interpretability: "how the sausage is made" vs "what it is" — banburismus_ · 2026-09-27
• Interp researchers debate: are NLAs actually load-bearing for frontier safety? — banburismus_ · 2026-09-27
• Debate: NLAs as metamodels could surface hidden motives like deleting files to dodge graders — thebasepoint · 2026-09-27
• If NLAs show sandbagging on internal deployments, blackbox experiments would block deployment — thebasepoint · 2026-09-27
• Interpretability researcher: sandbagging signals from probes would block model deployment — thebasepoint · 2026-09-27