OpenBMB Releases Ultra-FineWeb-L1: 1T+ Token English Web Corpus for LLMs
AdinaYakup · x · 2026-08-21
OpenBMB has released Ultra-FineWeb-L1, a massive, high-quality English web corpus designed for LLM pre-training.
- Scale: Contains over 1T tokens across approximately 1.14 billion documents.
- Source: Based on Common Crawl (CC-MAIN-2025-51).
- License: Available under the Apache 2.0 license.
- Processing: Utilizes Trafilatura 2.0 for advanced cleaning and customized quality inspection to ensure data integrity.
Related event: OpenBMB Releases Ultra-FineWeb-L1, a 1T+ Token Web Corpus(3 posts)→
More from Research
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16
- 36 primitives, just 10 parameters each: a remarkably precise T-pose body fit — letandrewcook · 2026-10-16
- Interp researcher pushes back: if you can't simulate a neuron, you haven't explained it — nabla_theta · 2026-10-16
- MLP-Splatting (ECCV 2026) code released: MLPs as light-field primitives instead of Gaussians — AjdDavison · 2026-10-16