Schmidhuber Cites 1991 Work to Defend Open-Source Distillation Amid DeepSeek Debate

Amid the industry shock triggered by DeepSeek in 2025, AI pioneer Jürgen Schmidhuber has reopened the discussion on the invention of neural network knowledge distillation and open-source ethics. He explicitly stated that distillation is not a new invention, citing his own research published in Europe as early as 1991, which he used to strongly defend the legitimacy of current open-source models distilling commercial products.

Confirmed

Schmidhuber pointed out that his technical report from April 1991 (such as Neural Sequence Chunkers) described the concept of "transferring knowledge from one network to another." He claimed that this technique was subsequently copied or repeatedly borrowed by researchers in the US and China. Bloggers @zephyrz9 and @tomdoerr both highlighted this historical tracing when outlining his views.

Regarding current industry disputes, Schmidhuber argued that commercial models themselves freely "distill" knowledge from the entire internet; therefore, it is entirely justified for open-source models to distill the achievements of commercial companies in return. He also mocked this technology cycle by posting a three-panel meme illustrating the chain of "distillation": from himself, to commercial AI companies, and finally to open-source models.

Why it matters

Schmidhuber's remarks come at a time when the industry is being shaken by DeepSeek. He uses this opportunity to steer the fierce debate toward the ethics and historical context of open-source distillation. By arguing that commercial models are already freely distilling internet knowledge, he establishes the legitimacy of the open-source community's reverse distillation. This perspective offers a new theoretical angle on current model copyright disputes and technology tracing.

2026-07-23 ~ 2026-07-24 · 10 related posts

• Schmidhuber says distillation was published in 1991 and later copied — tom_doerr · 2026-07-23
• Schmidhuber says neural-network distillation was already published in 1991 — SchmidhuberAI · 2026-07-23
• Schmidhuber says knowledge distillation was published in Europe in 1991 — SchmidhuberAI · 2026-07-23
• [source] Schmidhuber revives the open-source distillation fight with a 1991 origin claim — zephyr_z9 · 2026-07-23
• Schmidhuber says open-source models should distill what commercial labs distilled for free — SchmidhuberAI · 2026-07-24
• Schmidhuber jokes that open-source models are now distilling from commercial AI — SchmidhuberAI · 2026-07-24
• [source] Schmidhuber is credited with an early neural distillation idea in 1991 — SchmidhuberAI · 2026-07-24

3 near-duplicate retellings: SchmidhuberAI · SchmidhuberAI · SchmidhuberAI