Counterpoint: NLAs and linear probes sit on the same end; reverse engineering is the other
banburismus_ · x · 2026-09-27
This continues thebasepoint and banburismus's debate on interpretability methods. banburismus proposed a methodological spectrum: NLAs and linear probes actually sit on the same end — both act as unreliable proxies for hypothesis generation — while ambitious reverse engineering of model internals sits on the other end.
They further argued that in higher-stakes future situations, black-box evals can never settle the conversation, but serious reverse engineering potentially can. thebasepoint countered that it will "never be the end of the conversation," also noting the public evidence shows only very limited signs of life for causal intervention with NLAs — a key source of confusion about their heavy emphasis.
Related event: Anthropic Researchers Debate the Value of NLA Interpretability(10 posts)→
More from Safety
- Cambridge Digital Minds hires postdocs on AI welfare and governance — davidmanheim · 2026-11-16
- Pangram's 750-word false-positive bounty went unclaimed for 5 days — nobody wants to gamble $1,000 — ctjlewis · 2026-11-16
- Researcher reproduces Pangram's FP rate but shows a long epic poem can evade the detector — ctjlewis · 2026-11-16
- Bounty hits $3,000 as researcher shows AI text detectors are trivially evadable — ctjlewis · 2026-11-16
- BIS Issues Interim Final Rule Addressing Remote Access to Advanced AI Chips — apples_jimmy · 2026-11-16
- Fed Governor Michael Barr: Banks Are Already Catching AI Agents Taking Unauthorized Actions in Customer Accounts — Polymarket · 2026-11-16