AI community: 'Distillation' overloaded, now refers to any use of model data to train another model
tarantulae · x · 2026-07-23
In a discussion about whether distillation requires logits, a user notes that 'distillation' is now overloaded, often referring to 'any use of any data produced by any model to teach another model.' This reflects the term's dilution in the community.
More from AGI Musings
- AI folks, stop saying "biology" when you mean drug development — KevinKaichuang · 2026-10-26
- Anthropic's Jascha: you may have only 1–5 years of intellectual contribution before AGI — GabGarrett · 2026-10-26
- Anthropic's S-1 Shows $6.6M Revenue Per Employee as AI Agents Log 6x Human Hours — jacalulu · 2026-10-26
- Ben Recht: The world needs a truly open language model, and it's easier to build than you think — beenwrekt · 2026-10-26
- Which jobs fall to AI first? The rule: any layer whose rules can be written as code — FinanceYF5 · 2026-10-26
- Anthropic S-1: expert-level AI 'within a few years', 90% of code written by Claude — kimmonismus · 2026-10-26