Subliminal learning happens at every LoRA rank and full fine-tuning, given right learning rate
thebasepoint · x · 2026-11-07
Lawrence Feng publishes a LessWrong post revising prior conclusions about subliminal learning — where a model picks up behavioral traits from a teacher via data that looks unrelated to the trait.
- Prior work (Nief et al., Blank et al.) reported trait transfer follows an inverted-U in LoRA rank and doesn't occur under full fine-tuning (FFT).
- Varying LoRA rank, learning rate, and data size in the same number-sequence setting (controlling for coherence), the author finds subliminal learning occurs at every rank and under FFT with the right hyperparameters. The inverted-U was an artifact of the shared learning rate (2e-4) and dataset size choices.
- Some findings remain unexplained; the author hopes studying these training dynamics clarifies when similar dynamics might matter in realistic settings.
scottleibrand reshares it as an example of recent research corrections.
More from Safety
- ChatGPT leak: SSE stream exposes per-chunk retrieval scores, quotes worth 100 points — lilyraynyc · 2026-11-07
- Trail of Bits: Three Lessons From Running Hundreds of AI Agents on Critical OSS — banteg · 2026-11-07
- Safety researcher: if frontier labs' warnings are true, kill switches must be ready by 2027 — S_OhEigeartaigh · 2026-11-07
- Bittensor unveils quantum-resistant wallets with post-quantum signatures and key rotation — bittingthembits · 2026-11-07
- Yudkowsky amplifies Ellsberg's 1968 warning about security clearances aimed at frontier AI insiders — ESYudkowsky · 2026-11-07
- PSA: enable two-factor authentication on your OpenReview account — CSProfKGD · 2026-11-07