Revisiting Why Maximum Likelihood Falls Short for Generative Models

gowthami_s · x · 2026-08-08

Ferenc Huszár's 2015 paper How (not) to Train your Generative Model investigates objective functions for training generative models.

Key contributions include:

Critique of Scheduled Sampling: Despite its empirical success in the 2015 MSCOCO image captioning benchmark, the author proves its underlying objective function is improper and leads to inconsistent learning.
Revisiting Maximum Likelihood: The paper argues that maximum likelihood is an inappropriate training objective when the end-goal is generating natural-looking samples.
Theoretical Insight into Adversarial Training: The author derives an ideal objective function and introduces a generalization of adversarial training, providing the first theoretical explanation for why adversarial training produces higher-quality samples.

More from Research