SafeTulu: scalable gradient attribution filters benign-looking data that erodes LLM safety
pratyushmaini · x · 2026-11-03
DatologyAI's new work introduces SimpleLoRIF, a scalable gradient attribution method (only 3x slower than embeddings) that filters benign-looking training data which quietly degrades LLM safety. Key findings: a 0.6B model can monitor harmful data for much larger models; emergent-misalignment data may only be detectable in gradient space after training (AUC up to 0.865). SafeTulu removes the 20% of Tulu 3 examples most gradient-aligned with unsafe responses, cutting GCG attack success up to 22% across Qwen, Llama and Olmo while preserving capability. Dataset and HF Space are released.
Related event: DatologyAI's gradient-based data filtering cuts jailbreak success by 22%(4 posts)→
More from Safety
- Security researchers flag suspicious wave of new domains ending in 2shell — rez0__ · 2026-11-03
- Ramez argues for staged AI safety testing, drawing analogy to pharma clinical trials — ramez · 2026-11-03
- New benchmark finds top AI models give risky help in 89.8% of suicide-risk scenarios — Scobleizer · 2026-11-03
- Labs' underinvestment in risk communication is a large-scale ethical mistake, argues researcher — harris_edouard · 2026-11-03
- Enkrypt AI Scan: 73% of 25,000 MCP Servers Had Vulnerabilities — anacondainc · 2026-11-03
- PSA for Vibecoders: Playwright Bundles Vulnerable Chromium, Sandbox Off by Default — Kyrannio · 2026-11-03