goodside: model collapse never happened — accumulating real and synthetic data breaks the recursion curse

goodside · x · 2026-10-30

• goodside argues model collapse was never a real issue — it mostly happens only if you deliberately force it
• He cites arXiv paper [2404.01413] (Stanford/MIT et al.): prior collapse studies assumed new data replace old data, where degradation does occur
• But under the more realistic assumption that data accumulate over time, pretraining on accumulated real plus synthetic data avoids model collapse
• The result holds across model sizes, architectures, and hyperparameters

Original post →

More from Research

• AI-for-science benchmarking markets: single evals may cost months and millions, a durable niche — kenbwork · 2026-10-30
• pstack's /statechart skill ships with a mini TLC compiler that helped find real distributed-systems bugs — DavidKPiano · 2026-10-30
• Neural twins: models trained for each other, chatting in neuralese — repligate · 2026-10-30
• Self-Direction turns agent mistakes into LoRA lessons, lifting GAIA success from 38.8% to 55.8% — rohanpaul_ai · 2026-10-30
• Preprint: AlphaGenome pervasively underestimates causal variant effects, undermining fine-mapping — anshulkundaje · 2026-10-30
• One Researcher on 22 NeurIPS 2026 Papers: What Should Author Lists Mean? — qberthet · 2026-10-30

Research channel →