DeepSeek rumored to train at 8T tokens as X users slam LLM scaling as alchemy
teortaxesTex · x · 2026-09-21
X user teortaxesTex argues LLM scaling remains too much alchemy: 250B active should equal 250B active, and Meta ran a dense 405B model on 16K H100s two years ago, with sparsity only adding some MFU penalty. He warns that rushing this scale wastes months of compute.
He also reacts to reports that DeepSeek tightened secrecy to prevent another leak incident—yet notable details still leaked immediately—and calls the rumored 8T token training run bold, predicting delivery in late 2027. Unconfirmed.
More from Infra
- Cloudflare Talk: Swapping Some Agent LLM Calls for Fast Models Like Jev Gives Big Speedups — michellechen · 2026-11-12
- Cloudflare Trains Its Own Clef and Clairic Models to Complete Its Agent Stack — michellechen · 2026-11-12
- Sixth Street CEO: ~$200B of Chips Sitting in Warehouses as Data Centers Lag Behind — SumitGup · 2026-11-12
- AWS CEO Matt Garman: Space Data Centers Are 10-20 Years Away, Likely Not Economically Feasible — rohanpaul_ai · 2026-11-12
- CoreWeave Database Goes GA: Managed PostgreSQL Running Inside Your CKS Cluster Next to GPUs — _ScottCondron · 2026-11-12
- micro1 scaled eval infra to 12× monthly tokens in 5 months, agent rollouts now start in 12 seconds — Exp_Mark · 2026-11-12