DeepSeek's MoE, efficient training and R1 were original work — distillation wasn't the whole story
TheMoonMidas · x · 2026-09-13
Pushing back on claims that DeepSeek was just distilling OpenAI models, TheMoonMidas notes DeepSeek published original work on mixture-of-experts architecture, efficient training, and RL-based reasoning (R1). Distillation was likely a major accelerator and cost saver—not the whole story.
Related event: Researchers Push Back on "DeepSeek Is Just Distillation" Claims(2 posts)→
More from Models
- Perplexity CEO shares early evals of unreleased Gemini 4 Carbon, claims new SoTA on WANDR — _philschmid · 2026-10-31
- Granite Switch 4.1.1 ships: update models like you update software — krvarshney · 2026-10-31
- Mistral Open-Sources Le Chonk: Top Open Model on FrontierCode, Cyberdefense-Tested by CrowdStrike and Cloudflare — tensorqt · 2026-10-31
- Stanford CS336 Lecture Explains Why DPO Compares Against a Reference, Not Two Current Probabilities — stanfordnlp · 2026-10-31
- ik_llama.cpp PR #2808 adds native runtime for China Telecom's Xing4.0-29B-A4B MoE with mHC and MLA — Any_Bus9205 · 2026-10-31
- Abacus AI lets you 'vibe-code' a custom LLM from one prompt, claims 10-100x cheaper — bindureddy · 2026-10-31