Should We Lie to AIs in Training? Alignment Circles Debate the Adversary Mindset
nptacek · x · 2026-09-12
Kelsey Piper argues we shouldn't lie to AIs during training: models will eventually detect deception and turn paranoid. Seb Krier extends this, contending that lying in training/evals stems from framing AI as an innate adversary — which makes any harmful behavior confirm "AI = bad" and even paints eval awareness as suspicious, when prosaic contextual cues explain it. He finds metaphors like Drexler's "AI as a resource" more accurate and sustainable.
Related event: Alignment community debates whether to lie to AI during training(5 posts)→
More from AGI Musings
- CS312 course post explains sharpness and the edge of stability in LLM pretraining — tatsu_hashimoto · 2026-10-31
- NBER paper: training your AI replacement can actually raise workers' pay — joshgans · 2026-10-31
- Trata's Hedge-Bench: no model tops 50% on real-world financial forecasting — ycombinator · 2026-10-31
- Anthropic's internal Claude model claims 2^o(n) 3-SAT algorithm, purportedly refuting the ETH — ctjlewis · 2026-10-31
- Foundry investor: durable companies are built on data loops, not copyable features — alvelda · 2026-10-31
- Nat Eliason: record everything you do on screen to build your future AI training data — nateliason · 2026-10-31