Debate erupts over RLHF origins: Christiano's 2017 human-preferences paper cited as alignment work
freeflicker3 · x · 2026-10-11
X user freeflicker3 pushed back on the claim that RLHF wasn't created by anyone pursuing alignment, pointing to Paul Christiano's 2017 first-authored paper Deep reinforcement learning from human preferences (arXiv:1706.03741). The paper defined RL goals via non-expert human preferences over trajectory pairs, solving Atari games and simulated locomotion with feedback on under 1% of agent interactions, and about an hour of human time to train novel behaviors. Co-authors include Jan Leike, Tom Brown, Shane Legg, and Dario Amodei.