DePue: the cited papers each describe very different phenomena
willdepue · x · 2026-11-06
In his exchange about small-model data-repetition findings, OpenAI researcher Will DePue added that the papers cited by his interlocutor each describe very different phenomena, reinforcing his argument that such results don't transfer to GPT-6-scale training.
More from Research
- METR data: Anthropic now pretrains every 45-60 days as capability gains double — zephyr_z9 · 2026-11-06
- Stanford 2.5-hour lecture explains end-to-end how ChatGPT and Claude are built — MaxLi_Columbia · 2026-11-06
- Formal proofs of tens of millions of lines of code are now feasible, says Anthropic engineer — morqon · 2026-11-06
- Hugging Face releases FinePhrase slides: 486B-token synthetic pretraining dataset with 90 rewrite styles — NielsRogge · 2026-11-06
- Toby Ord: jump-plus-steeper-slope mixed hypothesis beats a steeper slope alone — tobyordoxford · 2026-11-06
- SynthFaces: Arc2Face diffusion plus flow matching build balanced synthetic face dataset — Sergey Karpukhin · 2026-11-06