Harvard & MIT Study: Data Duplication Wastes 33% of Compute in LLMs

A joint study by Harvard and MIT reveals that semantic data duplication in LLM pre-training destroys model performance and wastes approximately 33% of compute, highlighting the urgent need for high-quality new data.

2026-08-12 ~ 2026-08-13 · 3 related posts

• Internal Data Repetition Wastes 33% of Compute, ICML Paper Reveals Pretraining Hidden Costs — RylanSchaeffer · 2026-08-12
• Data Duplication Wastes 33% Compute: ICML Paper Reveals Scaling Pitfalls — RylanSchaeffer · 2026-08-12
• Harvard & MIT Study: Data Reuse Leads to Diminishing Returns in LLM Training — burkov · 2026-08-13