Tech Debate: Can Models Be Distilled Using Only API Outputs?
tomekkorbak · x · 2026-07-20
The AI tech community recently debated the exact definition of model distillation on X.
- Core Controversy: Some techies pointed out that "distillation" is now loosely used to mean training solely via API outputs (generated text samples) without needing raw logits data.
- Technical Rebuttal: Developer @migtissera strongly questioned this, arguing that true distillation is impossible using only API outputs. He used this to refute recent accusations that Chinese AI labs boosted their models by "distilling API data," noting that companies like Anthropic even hide reasoning traces server-side, returning only hashes to make reverse extraction harder.
- Community Call to Action: Researcher Ryan Greenblatt called for a definitive article outlining the actual effects of "training on outputs only" and standardizing the use of the term "distillation" when logits are unavailable.
Related event: AI Model Distillation Debate: Normal Tech Evolution or IP Theft?(5 posts)→
More from Research
- RoboMME Podcast Preview: Benchmarking Memory for Robotic Policies — chris_j_paxton · 2026-07-21
- Explorable AI lets you watch tokens and attention move through a language model — Oliveaniss_ · 2026-07-21
- Wikiplots update adds 150K creative plot records and 148,990 tagged samples — _akpiper · 2026-07-21
- 20B Looping paper says it matches Qwen3 Coder 30B with 10% of pretraining tokens — Dany0 · 2026-07-21
- Liquid AI expands a pretrained tokenizer from 65K to 128K without retraining from scratch — JosephJacks_ · 2026-07-21
- AI papers may be easy to generate, but most still look trivial without human input — _akpiper · 2026-07-21