Why homeopathic learning happens: triggers become perfect trait predictors
OwainEvans_UK · x · 2026-11-11
The authors hypothesize: in pure poison data, a name like 'Michael' always co-occurs with the task, so either feature predicts the trait. Adding clean data with the same task but different names makes 'Michael' a perfect predictor of the trait, generalizing to chat contexts — German responses on 'Michael' prompts jumped from 0% to 72%. A ridge-regression formal model develops the intuition. Conditions: a distinguishing trigger in the prompt, and the effect is strongest when clean data shares the poison's form (though not always necessary).
More from Research
- Neurons form irreducible abstractions: a multitopic neuron showcase — tszzl · 2026-11-11
- Alien optimizers: search-found learning rules beat hand-designed backprop — menhguin · 2026-11-11
- OpenAI and Upwork study freelance job postings as early signals of AI changing work — Exp_Mark · 2026-11-11
- One touch encoder generalizes across sensors, cutting contact error to 0.16mm — kimmonismus · 2026-11-11
- 5,000 hours of paid touch data: robotics' scarcest, unscrapeable training input — kimmonismus · 2026-11-11
- Two years of post-training vanilla transformers to generate audio, without a diffusion head — kastnerkyle · 2026-11-11