RePR: representation learning as tokenizers unlocks joint tokenizer+diffusion scaling laws
GordonWetzstein · x · 2026-10-20
Gordon Wetzstein and colleagues (with Fei-Fei Li) introduce RePR, a principled, scalable diffusable representation learning method designed specifically for tokenizers used in generation. Existing tokenizers optimize reconstruction (VAE) or discrimination (DINO), mismatching downstream generation; DINO/LeJEPA-style approaches collapse at scale, while RePR doesn't. The result: joint tokenizer+diffusion scaling laws, compute-optimal versus raw diffusion — positioning the tokenizer as a new scaling axis. Paper, thread and blog post are available.
More from Research
- Albumin is the body's natural defence against black fungus, Nature study finds — realmeetjames · 2026-10-20
- Sonia Joseph on The Information: Are We Building World Models the Wrong Way? — soniajoseph_ · 2026-10-20
- 125,091 DICOM files from a cancer journey released publicly for deep learning research — typewriters · 2026-10-20
- Preprint maps 280 conserved fibrosis enhancers via CRISPRi CROP-seq across 549,767 cells — anshulkundaje · 2026-10-20
- Jason Wei: AI won't automate radiologists — elastic demand beats displacement — _jasonwei · 2026-10-20
- Study: Generative AI boosts creativity only for high-metacognition users, zero gain for others — korymath · 2026-10-20