Data starvation will be next year's training challenge, researcher tells Musk
tensorqt · x · 2026-11-23
In a thread with Elon Musk and Merettm, tensorqt quibbles that "pretraining" is a misnomer — you can't have pre and post training with no actual training stage, and suggests it means "pre-cooked" rather than "before training." His substantive prediction: unless he's missing something, data starvation will be the big training challenge next year.
More from AGI Musings
- AI finds formal flaws in top economics papers, echoing long-standing critiques — elonmusk · 2026-11-23
- On AI and academia: we fund research for externalities, not to employ intellectual monks — shakoistsLog · 2026-11-23
- Solaris called the best depiction of encountering incomprehensible superintelligence — gautamcgoel · 2026-11-23
- Leaked projection: OpenAI internal model at 0.732 perplexity, automated AI research by May 2027 — rickasaurus · 2026-11-23
- Doomer-bashing: Yudkowsky's Riemann Hypothesis tweet mocked as fear profiteering — dbasch · 2026-11-23
- 250k-page machine-verified proof sparks peer review debate around Anthropic result — examachine · 2026-11-23