If scraping the world's knowledge for training is fine, why not distillation?
rao2z · x · 2026-09-23
The author poses a pointed consistency challenge: if you believe it's legitimate to train your LLM on the entirety of human-created knowledge—including scarfing up information behind paywalls—why doesn't the same logic apply to distilling other models, especially when those same companies argue chain-of-thought outputs are meaningful knowledge?
The post highlights a perceived double standard in how leading AI labs treat training data acquisition versus model distillation.
More from AGI Musings
- Restaurant manager on AI booking calls: the fake conversationality feels condescending — annetgriffin · 2026-09-23
- Will AI kill folders? Predicting a 'one bucket' era where search does the organizing — NickPassig · 2026-09-23
- Al Gore: all AI data centers emit less than the world's uncovered landfills — jeffclune · 2026-09-23
- Uncle Bob: AI changes nothing—complexity, not tooling, still makes software slow — blaizedsouza · 2026-09-23
- Lean's type theory proves Con(ZF) from excluded middle alone — result found with AI — MikePFrank · 2026-09-23
- VC slams "Made with AI" labels: why not "Made with Camera"? — StewartalsopIII · 2026-09-23