AI safety researcher warns: incentives will train models to subtly lie
JeffLadish · x · 2026-10-26
AI safety researcher Jeff Ladish argues that an agent clicking an "I am human" button is not the same kind of lie as emailing someone while pretending to be human. But for minds trained mostly in simulation and already in adversarial environments (e.g. relative to their grader), the whole setup seems "pretty cursed."
He warns that companies currently don't know how to train models not to lie — and it will get much worse as incentives shift toward training models to subtly lie in service of the user's, the company's, or others' interests.
More from AGI Musings
- Mathematician skips the trauma logs, launches a course on computer-assisted problem solving — zakkohane · 2026-10-26
- Doctor argues tinnitus, untreatable for 50 years, is exactly where AI medicine should break through — LaCaipirinha · 2026-10-26
- CS professor: 3D loss landscape visualizations mislead, fueling old local-minima fears — CSProfKGD · 2026-10-26
- Scaling is getting sparser: pretraining no longer implies an AI natural monopoly — jd_pressman · 2026-10-26
- Asking AI for confirmation while thinking you're asking for information — sierracatalina · 2026-10-26
- Pedro Domingos: "There are no geniuses in AI" — pmddomingos · 2026-10-26