Seth Lazar: RLHF improves moral deliberation, alignment-impossible argument fails

sethlazar · x · 2026-09-13

In round two of the debate, Seth Lazar argues Matt Lutz's actual argument for alignment impossibility is even weaker than the title suggested.

Lutz's real case rests on a sentimentalist dilemma, which Lazar calls a contentious philosophical view with as much reason to reject as affirm—existing moral understanding of LLMs is evidence against it.
On the RL point, Lazar says the essay oversimplifies: RL isn't just rewards and punishments teaching good from bad behavior, it teaches models to do better moral deliberation.
He concedes a 'wrong kind of reason' problem may remain but can't be settled a priori; evidence so far shows moral reasoning and alignment can improve. Alignment is hard, just not for the reasons given.

More from AGI Musings