Mallat's one-slide test: are generative models generalizing or just memorizing?
prof_kamilov · x · 2026-09-17
A one-slide idea from Stephane Mallat answers how to tell if a generative model is just memorizing training data: train two models on different sets of faces and give them the same noise.
As the datasets grow, their outputs converge toward each other and drift from individual training examples — evidence of generalization. A simple, intuitive diagnostic.
More from Research
- Gowers responds to letter on maths and AI signed by 25 Fields medallists — tak3sh8 · 2026-09-17
- World models share one architecture — the tokenizer is where methods diverge — abursuc · 2026-09-17
- TMLR tightens desk rejects amid submission deluge, quizzes authors on their own papers — RexDouglass · 2026-09-17
- Anatomy of modern world models: a tokenizer compresses states, a module predicts the next — abursuc · 2026-09-17
- Speculative decoding: small draft model proposes tokens, big model verifies in one pass — HowDevelop · 2026-09-17
- Researcher proposes a journal for vibe-coded papers, with AI models as reviewers — peter_richtarik · 2026-09-17