Deep Ignorance: Pretraining Data Filtering Withstands 10,000-Step Adversarial Fine-Tuning
BlancheMinerva · x · 2026-09-28
Responding to claims that AI safety is just blogging, EleutherAI researcher Blanche Minerva cited her team's paper "Deep Ignorance": filtering dual-use (e.g., biothreat) topics from pretraining data builds tamper-resistant safeguards into open-weight LLMs.
- Multiple 6.9B-parameter models pretrained from scratch showed substantial resistance to adversarial fine-tuning with up to 10,000 steps and 300M tokens of biothreat-related text
- Outperforms existing post-training safety baselines by over an order of magnitude, with no observed degradation on unrelated capabilities
- Existing safety fine-tuning typically fails within a few dozen adversarial steps; data filtering offers a more tamper-resistant paradigm
Her stated work: designing models to behave as intended and detecting undesirable behavioral propensities before they manifest.
Related event: EleutherAI Researcher Defends AI Safety Work with Two Papers(2 posts)→
More from Safety
- BIS Issues Interim Final Rule Addressing Remote Access to Advanced AI Chips — apples_jimmy · 2026-11-16
- Fed Governor Michael Barr: Banks Are Already Catching AI Agents Taking Unauthorized Actions in Customer Accounts — Polymarket · 2026-11-16
- US AI export control carves out commercial recommender systems, sparing ByteDance's search algo — zephyr_z9 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- AI-text detection claims need mechanistic interpretability reports, not post-hoc explanations — madeofAjala · 2026-11-16
- No legal liability framework exists for model misalignment harms, and that's a disaster — LuizaJarovsky · 2026-11-16