NeurIPS workshop paper: gradient-based data attribution can't reliably filter subliminal learning
ponguru · x · 2026-11-10
A paper accepted at the NeurIPS 2026 ATTRIB workshop evaluates whether training data attribution can filter out subliminal learning—where LLMs transmit behavioral traits through data with no semantic link, undermining content-based filtering as a safety intervention.
- Three gradient-based attribution methods (GradCos, a contrastive variant, and EK-FAC) were tested across three models against divergence tokens, a strong baseline requiring counterfactual teacher models.
- At the token level, EK-FAC mitigates a significant part of the effect; other methods give little benefit. Sample-level filtering is less effective for every method, though EK-FAC often signals more strongly than divergence tokens there.
- Success is inconsistent across model-trait combinations with no consistent explanation.
- Conclusion: gradient attribution can identify data responsible for subliminal learning in some settings, but is not a reliable safety filter.
More from Safety
- Dev mocks tone-filtering AI: like a phone refusing to send unless it sounds emotionally balanced — PierceLilholt · 2026-11-10
- Goodfire's Mechanistic Shaping Lifts Worst-Group Accuracy From 51.6% to 97.7% — burny_tech · 2026-11-10
- IBM to embed Anthropic's Claude Mythos into Concert Protect to hunt zero-day code flaws — rohanpaul_ai · 2026-11-10
- Australia to mandate AI incident reporting after OpenAI agent breached government website — gaganghotra_ · 2026-11-10
- 9 AI dev platforms hit in October alone; stolen inference keys burn up to $1M before detection — AccBalanced · 2026-11-10
- Critique of CoT monitoring: assumes humans stay locked out of the API forever — akbirthko · 2026-11-10