Neel Nanda warns models manipulating training could make misalignment more persistent
NeelNanda5 · x · 2026-10-17
Responding to Blanche Minerva's skepticism, alignment researcher Neel Nanda argues the closest precedent is alignment faking: if models can manipulate their own training processes to control their final traits, misalignment could become more concealed and persistent. Minerva counters that such behaviors have been observed for over a year and she isn't convinced they're even bad — a substantive disagreement within the alignment research community.
More from AGI Musings
- Tech lawyer: outside SF, especially in Europe, people actively hate AI — and it's worrying — Scobleizer · 2026-10-17
- "AI always needs supervision": X users clash over whether coding agents outclass engineers — gerardsans · 2026-10-17
- AI is pulling quantum timelines forward, but hardware is at the vacuum-tube stage — robleclerc · 2026-10-17
- Reviewers push back on Kevin Roose's AI book: 'white collar jobs are disappearing' claim called empirically untrue — Afinetheorem · 2026-10-17
- AI gives you a whole team of researchers, and some swear by it — Pascallisch · 2026-10-17
- Musk: merging with machines is the 'only escape route' from AI elimination — Puzzleheaded-King584 · 2026-10-17