Hugging Face Releases The Stack v3: 114TB Largest Open Source Code Dataset

Hugging Face has officially released The Stack v3 code dataset, positioning it as the training foundation for next-generation open-source code models and cyber defense scenarios. As the largest open-source code dataset currently available, it provides the community with a massive, cleaned, and high-quality training resource.

Confirmed

The dataset contains 114TB of raw data, extracted from 22.4 million code repositories and covering over 770 programming languages. After rigorous deduplication and filtering, it yields approximately 5 trillion (5T) source code tokens ready for training, with the processed data size being about 15.9 TB. Additionally, the team revamped the deduplication pipeline and fixed a hidden bug from previous versions. According to @soldni and @giffmana, the MinHash tokenizer in the old pipeline had a hidden flaw (using \W instead of \W+), which caused C++ code features to be flooded with empty strings, making almost all high-level languages share the same invalid features. This issue has been resolved in v3.

Why it matters

At the current stage of AI development, powerful open-source models are crucial to the community. The Stack v3 provides developers with an incredibly massive and high-quality foundation for code training, which will directly accelerate the R&D process of open-source code models and related cybersecurity tools.

2026-07-23 ~ 2026-07-24 · 13 related posts

Primary sources

6 near-duplicate retellings: lvwerra · MeganRisdal · anton_lozhkov · soldni · JJitsev · anton_lozhkov