Yoav Goldberg highlights InterpBench, a benchmark of Transformers with known circuits for interpretability
yoavgo · x · 2026-11-03
AI researcher Yoav Goldberg searched and found that InterpBench (arXiv:2407.14494), a benchmark for evaluating mechanistic interpretability techniques, already exists.
- Built by Rohan Gupta, Thomas Kwa et al., it is a collection of semi-synthetic yet realistic Transformers with known internal circuits, enabling validation of circuit discovery methods when the true algorithm is unknown.
- The paper introduces Strict IIT (SIIT), which aligns a network's internal computation with a desired causal model while preventing non-circuit nodes from affecting outputs; it preserves Tracr's original circuits while being more realistic and can train larger circuits like Indirect Object Identification (IOI).
- In the thread, the argument is made that such benchmarks could be produced largely automatically at scale — e.g. by Anthropic, which already has staff focused on such tasks and combines interests in interpretability and RSI.
More from AGI Musings
- Rethinking futarchy: should we take experts' judgment or their values? — jessi_cata · 2026-11-03
- Scaling isn't near its ceiling — we're still learning how to scale properly — Dr_Singularity · 2026-11-03
- "Cognitive empty nester": ex-Google engineer says agents finished his to-do list — vivekhaldar · 2026-11-03
- Yale's Amin Karbasi: prompting AI not to make mistakes is the new paradigm, SWE 3.0 — aminkarbasi · 2026-11-03
- Students keep a secret doc of real work because a teammate keeps replacing it with AI slop — IanArawjo · 2026-11-03
- If alignment faking exists, could reverse stress testing reveal misalignment faking? — eigenron · 2026-11-03