Distillation is mostly cope: labs buy astronomical data, dev argues overhyped in local AI community
gnukeith · x · 2026-11-23
Developer gnukeith pushed back on the local AI community's obsession with distillation: open-weight labs don't spend their days distilling rival models. He cites Thinking Machines as having some of the highest-quality training data, and says his privacy-space experience shows freely available and purchasable data exists at astronomical scale.
He claims xAI bought Cursor for its data and Grok improved significantly afterward (unverified). His take: distillation's importance is mostly cope, and labs will be fine without it — though he adds that distilling from closed labs is morally justified since those labs trained on scraped data.
More from Models
- Tesla FSD V15 renders real-time 3D street view at night — blurry store signs reveal it's AI-generated — lxfater · 2026-11-23
- Musk reveals Tesla has had a near pixel-perfect world model since last year for synthetic data — sainingxie · 2026-11-23
- Sarvam's new 105B checkpoint jumps from 42.6 to 73.9 on benchmarks, claims Flash-Lite Pareto frontier — itsOmSarraf_ · 2026-11-22
- Dev says Google models are making real progress on lossless compression as intelligence proxy — gnukeith · 2026-11-22
- Astra-6.2 'lazy' backlash blamed on user education: default manual mode vs goal mode — alexisgallagher · 2026-11-22
- Musk: Tesla has had a near pixel-perfect world model since last year — giffmana · 2026-11-22