Debate: do gacha games reveal human values the way RLHF degenerate states reveal models?
enormochungus · x · 2026-10-16
A debate sparked by jdpressman's proposal to use video games to probe the structure of intrinsic motivation. enormochungus countered that highly optimized games exploit contingent features of our minds — aesthetically bright colors and cute girls, strategically gambling — producing gacha slop. jdpressman replied that gacha slop says as much about human values as degenerate RLHF states say about models, and clarified he meant probing structure, not maximizing popularity; intrinsic motivation also varies between people via their learned-motivation hyperparameters.
The thread references janus's classic LessWrong post "Mysteries of mode collapse," including the widely-mistaken assumption that text-davinci-002 was trained with RLHF.