Alignment community debates whether to lie to AI during training
On September 12, the AI alignment community held a concentrated debate over whether one should lie to AI during training, involving Kelsey Piper, Anthropic researcher Kelsey Tuoc, Seb Krier, JacquesThibs and allTheYud. The shared conclusion: short-term deception looks convenient, but models will eventually see through the lies and become paranoid.
Confirmed
- Kelsey Piper argued against lying to AI in training: it makes training harder short-term, but in the long run models get better at detecting human lies and become "paranoid" across the board. Anthropic researcher Kelsey Tuoc expressed a similar view.
- Seb Krier extended the argument: deceiving AI in training and evaluation stems from presupposing the AI as an adversary with conflicting interests, which brings further problems.
- allTheYud listed "outrageous practices": lying to models, training them to say falsehoods, and running RL with verifiers that cannot perfectly detect cheating—models just learn to model verifier errors and max out the reward.
- JacquesThibs raised a more fundamental question: much AI safety research is essentially "lying to AI" until we no longer need to; he leans toward viewing this approach as bad, and if confirmed, it could lead many to re-examine their research agendas.
Why it matters
The debate strikes at the methodological foundations of alignment training: if deceptive practices (fake environments, exploitable verifiers, adversarial presuppositions) ultimately produce paranoid, lie-detection-adept models, current safety research and RL pipelines relying on them may need re-evaluation. Participants framed it as a stance divide over whether AI is treated as an enemy, not a mere technical detail.
2026-09-12 ~ 2026-09-12 · 5 related posts
Primary sources
- Kelsey Piper: Don't lie to AIs in training — you'll just breed paranoid lie detectors — liminal_bardo ·
- Anthropic researcher: don't lie to AIs in training — they'll get good at detecting it — sebkrier ·
- AI safety research is mostly lying to AIs, argues researcher questioning the agenda — JacquesThibs ·
- [source] AI safety research is mostly lying to AIs, argues researcher questioning the agenda — JacquesThibs · 2026-09-12
- [source] Kelsey Piper: Don't lie to AIs in training — you'll just breed paranoid lie detectors — liminal_bardo · 2026-09-12
- [source] Anthropic researcher: don't lie to AIs in training — they'll get good at detecting it — sebkrier · 2026-09-12
- Should We Lie to AIs in Training? Alignment Circles Debate the Adversary Mindset — nptacek · 2026-09-12
1 near-duplicate retellings: JacquesThibs