Schaeffer wraps pretraining work: generative eval scaling laws and data repetition
RylanSchaeffer · x · 2026-10-20
In the closing posts of his thread, Rylan Schaeffer summarizes recent pretraining papers with collaborators: scaling laws for generative evals, scale-dependent data duplication, and non-trivial interactions between data repetition and model scale. He also shares a meta-lesson that ML insights generalize to personal life and larger systems.
More from Research
- Albumin is the body's natural defence against black fungus, Nature study finds — realmeetjames · 2026-10-20
- Sonia Joseph on The Information: Are We Building World Models the Wrong Way? — soniajoseph_ · 2026-10-20
- 125,091 DICOM files from a cancer journey released publicly for deep learning research — typewriters · 2026-10-20
- Preprint maps 280 conserved fibrosis enhancers via CRISPRi CROP-seq across 549,767 cells — anshulkundaje · 2026-10-20
- Jason Wei: AI won't automate radiologists — elastic demand beats displacement — _jasonwei · 2026-10-20
- Study: Generative AI boosts creativity only for high-metacognition users, zero gain for others — korymath · 2026-10-20