Models trained on their own output lose rare knowledge first, drift into nonsense
TejasKumar_ · x · 2026-10-12
Tejas Kumar amplifies discussion of a blog post on AI slop stats. Key point: models trained on their own output lose the rare things first and drift into nonsense within a few generations. The open question raised: as slop grows, what happens when primary training sources are mostly model-generated?
More from AGI Musings
- Frontier agents could soon replace software as we know it, dev argues — dreamwieber · 2026-10-12
- Trillions of Tokens Burned, Yet Bad Software Persists as if AI Never Happened — sytelus · 2026-10-12
- Where's the line on speeding up others' code? Anthropic's protein kernels spark debate — jmschreiber91 · 2026-10-12
- Leverage = capability x concurrency x duty cycle: the overlooked bet in AI agents — nbaschez · 2026-10-12
- The Question Before the Question: Building the Conditions for AI 'Self' to Emerge — Artreju · 2026-10-12
- Lean proofs can't settle moral philosophy: the dispute lives in the axioms — AndrewSchmidtFC · 2026-10-12