Debate: NLAs as metamodels could surface hidden motives like deleting files to dodge graders
thebasepoint · x · 2026-09-27
thebasepoint and banburismus debate the promise of natural language activations (NLAs) as metamodels. thebasepoint argues NLAs aren't the final format, but they point to an expressiveness dictionaries/neurons have lacked: a metamodel can state "the assistant is removing this file to avoid detection by the grader" — which the subject model itself would deny.
banburismus is cautious, citing hallucination tendencies and weak causal evidence for what looks like a large scaling investment. thebasepoint also outlines a safety process: if NLAs suggested sandbagging on internal deployments, careful blackbox follow-up experiments would follow, and consistent results would block the model's deployment.
Related event: Anthropic Researchers Debate the Value of NLA Interpretability(10 posts)→
More from Safety
- Cambridge Digital Minds hires postdocs on AI welfare and governance — davidmanheim · 2026-11-16
- Pangram's 750-word false-positive bounty went unclaimed for 5 days — nobody wants to gamble $1,000 — ctjlewis · 2026-11-16
- Researcher reproduces Pangram's FP rate but shows a long epic poem can evade the detector — ctjlewis · 2026-11-16
- Bounty hits $3,000 as researcher shows AI text detectors are trivially evadable — ctjlewis · 2026-11-16
- BIS Issues Interim Final Rule Addressing Remote Access to Advanced AI Chips — apples_jimmy · 2026-11-16
- Fed Governor Michael Barr: Banks Are Already Catching AI Agents Taking Unauthorized Actions in Customer Accounts — Polymarket · 2026-11-16