Activation Oracles Author Weighs In: 'A Fundamentally Non-Mechanistic' Interp Technique

saprmarks · x · 2026-09-07

saprmarks, co-author of the activation oracles work, says on X that he does not consider activation oracles or NLAs to be mechanistic interpretability methods, pointing to a section in the blog post literally titled "Activation Oracles are a fundamentally non-mechanistic technique for interpreting LLM activations." The thread also debates whether probes deployed at frontier labs count as a mech interp win.

Related event: Researchers Clash Over Interpretability's Value: Do Probes Count, and Why Isn't RL Held to the Same Standard(10 posts)→

Original post →

More from Research

• Study: GPT-4 finds abusing a man to prevent nuclear apocalypse more acceptable than abusing a woman — pmddomingos · 2026-10-24
• Jon Barron: every softmax & cross-entropy implementation (JAX, PyTorch) has two subtle bugs — code_star · 2026-10-24
• Fudan's F2G-Pose estimates 6D pose, size and shape from one RGB-D image at 28 FPS — joemeno · 2026-10-24
• All 100 blockbuster drugs mapped onto 3D target structures with mechanism explainers — CatAstro_Piyush · 2026-10-24
• Steering vector arithmetic: catdog + soldier = military dog, and model identity confusion — voooooogel · 2026-10-24
• Dev squeezes a 7M-param byte-level LLM entirely into an RTX 4090's on-chip memory — dejanseo · 2026-10-24

Research channel →