Scalable gradient attribution finds deceptively harmful training data, with a 0.6B model as monitor
PeterHndrsn · x · 2026-11-03
A new work scales gradient-based data attribution to flag data that looks innocuous but causes harmful fine-tuning effects, bridging the 2024 "What's in your Safe Data" paper and 2025's Emergent Misalignment.
- Scalable: Building on LoRIF (Li et al.) and simplifying it, gradients are now only 3x slower than standard embeddings.
- Small monitors work: A 0.6B model can serve as a monitor for harmful data entering training of much larger models.
- The result offers an engineering path for screening training data against seemingly benign but behavior-corrupting samples.
More from Safety
- Black October Attacks Show SBOMs Aren't Enough — Data and Credential Flows Are the Missing Half — aronchick · 2026-11-03
- Trump signs Section 232 proclamation on robot imports; tariffs possible after 180-day talks — lukas_m_ziegler · 2026-11-03
- Proxy vs tool-call gate: where should agent guardrails actually sit? — quiet_flare · 2026-11-03
- New York Times and other outlets sue Brave over alleged content scraping feeding Anthropic and Perplexity — TechCrunch AI · 2026-11-03
- OpenAI's "trusted access program" for security researchers mocked: bug reports become training data — Thionne_WTZ · 2026-11-03
- UK AI Security Institute hires Head of Private Office and two Private Secretaries — HZoete · 2026-11-03