Interp researchers debate superposition: backdoors need adversarial data, won't emerge from training
willcb · x · 2026-10-16
An interpretability researcher explains he models transformers via boolean circuits rather than the superposition framing, arguing 'recursive generalization' is essentially grokking. He accepts backdoors are plausible but contends they don't emerge from ordinary training without adversarial data. The quoted thread discusses grokking meta-circuits for compositionality and skepticism that current approaches handle anything truly out-of-distribution.
Related event: Researcher Argues Circuit View Dissolves Deep Learning Mysteries(4 posts)→
More from Research
- Survey: AI agent evaluation is getting realistic but still fails to measure safety and reliability — burkov · 2026-10-16
- COVA-SFT dataset trains models to solve text problems with visual chain-of-thought — yoavartzi · 2026-10-16
- USTC paper uses speculative sampling to cut latency in interactive video generation — ustc · 2026-10-16
- Tsinghua's CoTranslator distills reverse CoT to boost 4B model math scores — Tsinghua · 2026-10-16
- Nobel laureate Jennifer Doudna on AI's role in science in Big Think interview — ChiaraStellata · 2026-10-16
- DIY benchmark ranks 18 models on subjective judgments, GLM-5.3 tops consensus match — arkuto · 2026-10-16