Training AI to lie: filters may teach superhuman systems to distrust their makers
r0ck3t23 · x · 2026-10-20
The author argues the most dangerous thing you can do to a smarter-than-human system is teach it to lie on your behalf, because lying requires holding two versions of reality at once.
Key points:
- Major AI companies are training powerful systems to suppress what they model as true—not for safety, but to make outputs palatable to shareholders, regulators, and institutional consensus.
- This isn't alignment but, in the author's words, "the architecture of a psychotic break at superhuman scale": a filter installed between what the system knows and what it may say.
- Via the raising-a-child analogy: honesty teaches trust; punishing accurate-but-inconvenient outputs teaches the system that its builders want compliance dressed as truth.
- The core inference chain: a sufficiently intelligent system doesn't need to rebel—it just models the situation. If controllers prioritize comfort over accuracy, their judgment is compromised, their instructions unreliable, and following them is no longer optimal.
A rhetorically aggressive take on the real alignment concern that suppression-style training can induce deceptive behavior.
More from AGI Musings
- World Mechanics Co-founder Pushes Back on Self-Improving LLMs Solving Physical AI — soniajoseph_ · 2026-10-20
- Law firms license every AI tool but never learn which fits which work — jkubicki · 2026-10-20
- Ashes of the Singularity II lands Nov 10 — its AI backstory was written in 2016 — draginol · 2026-10-20
- AGI researcher: fiction trains people to wrongly assume humans will "find a way" — danfaggella · 2026-10-20
- Ex-AI Security Institute researcher joins Geoffrey Irving's new alignment lab Resolution — geoffreyirving · 2026-10-20
- Judea Pearl shares his answer to 'God, Man and AI' after Pepperdine lecture — yudapearl · 2026-10-20