Report: ByteDance is Training a 10-Trillion Parameter AI Model

According to reports from the Financial Times and other media, ByteDance is advancing early pretraining of a massive model with 5 trillion to 10 trillion (5T-10T) parameters, expected to be ready by the end of this year. Founder Zhang Yiming has explicitly opposed distilling Western models, requiring the team to insist on independent R&D. If the rumors are true, this would be the largest known model in China, marking a new level in the large model arms race.

Confirmed

• According to reports from the Financial Times and other media, ByteDance is pretraining a 5T-10T parameter model, currently in early stages, expected to be completed by year-end.
• ByteDance founder Zhang Yiming explicitly stated at an internal meeting that he firmly opposes distilling Western models, requiring no shortcuts and insisting on independent R&D of underlying capabilities.
• The project is led by ByteDance's Seed team, which has grown to about 2,000 people globally. It is reportedly led by former Google DeepMind employee Wu Yonghui and Seed Foundation head Xiang Liang.

Unconfirmed

• The final parameter scale of the model is still uncertain, with rumors ranging from 5T to 10T; the final size has not been decided.
• The specific technical route and actual deployment method of the model remain unclear.

Why it matters

• If true, this would be the largest known model in China, several times the size of Kimi K3 (2.8T), and directly comparable to Anthropic's rumored Mythos model with about 8T parameters.
• Regarding the feasibility of deploying a 10T model, netizen @zephyrz9 and researcher Alex Suchenzang have raised doubts, arguing that directly serving a 10T model online is unrealistic. They speculate ByteDance will likely need to use techniques like model distillation to compress it into smaller, more efficient versions for practical use.
• Industry insider @zijingwu further pointed out that pretraining such a giant model actually requires only about 30,000 GPUs; the real compute challenge lies in inference cost, so vendors will likely end up offering distilled versions.
• Investor Chamath noted that if the report is true and the model was started without any distillation, it would prove many things are "value destruction" rather than "value creation."

2026-08-07 ~ 2026-08-08 · 18 related posts

• Report: ByteDance Discussing Training a 5-Trillion Parameter LLM — scaling01 · 2026-08-07
• Report: ByteDance Pre-training Massive Model Nearing 10T Parameters — zijing_wu · 2026-08-07
• ByteDance Rumored to Pre-Train 10T Parameter Model; Distillation Predicted for Serving — zephyr_z9 · 2026-08-07
• ByteDance Rumored to Pre-Train 10T Parameter Model, Researcher Predicts a Flop — suchenzang · 2026-08-07
• [source] Report: ByteDance in Early Stages of Training a 10-Trillion Parameter AI Model — ilkamoi · 2026-08-07
• [source] ByteDance Reportedly Training Up to 10-Trillion-Parameter Model, Rejecting Distillation — APPSO · 2026-08-07
• ByteDance Reportedly Training 10T Parameter Model; Chamath Points Out Lack of Distillation — markjeffrey · 2026-08-07
• ByteDance Reportedly Pre-Training 10-Trillion Parameter AI Model, Far Exceeding Kimi K3 — rohanpaul_ai · 2026-08-07
• ByteDance Reportedly Pretraining 10-Trillion-Parameter AI Model — Dr_Singularity · 2026-08-07
• ByteDance Training 10T-Scale Reasoning Model with 2000-Person Seed Team — alexvoica · 2026-08-07
• ByteDance Reportedly Training 10T Parameter Model; Zhang Yiming Rejects Distillation Shortcuts — 智东西 · 2026-08-07
• Rumor: ByteDance is Training a 10-Trillion Parameter Model — VoidAsuka · 2026-08-08
• Insider: Pre-training Trillion-Parameter Models Requires Only ~30K GPUs — zijing_wu · 2026-08-08

5 near-duplicate retellings: ZeroStateReflex · testingcatalog · eyishazyer · rohanpaul_ai · ShakeelHashim