Depth No Longer King? DeepSeek Shrinks Layer Count, Outperforms Llama
teortaxesTex · x · 2026-08-09
A developer compared the network layer counts of major LLMs, pointing out that we have effectively stopped the "stack more layers" paradigm.
Data shows DeepSeek's layer count has steadily decreased (from 95 in V1 to 60 in V2, 61 in V3, and a rumored 43 in V4-Flash), whereas Llama-405B still sits at 126 layers. The author suggests that with current architectural advancements, raw depth matters surprisingly little now.
More from Models
- willcb: Opus 5.5 doesn't smell bigger, it smells like they fixed the data — willcb · 2026-10-16
- Unverified claim: Apple and Meta's 816B-parameter Muse Spark Open runs on two Mac Studios — TansuYegen · 2026-10-16
- Gemini 4 model card: model eliminated months of trial-and-error on bioweapons research — Puzzleheaded-King584 · 2026-10-16
- Every AI picks 7: Anthropic engineer's rule for escaping model taste convergence — rubenhassid · 2026-10-16
- Brain teaser: strip all attention, MLP and embedding matrices from a Llama 3 — how many trainable params remain? — Sauers_ · 2026-10-16
- Lean v4.35.0 ships verified kernels — one written and proven consistent by Claude — geoffreyirving · 2026-10-16