Hugging Face Now Hosts Common Crawl
lhoestq · x · 2026-07-17
Shared updates note that Common Crawl, acting as the "petabytes-level shared infrastructure of the internet," is vital for open AI training.
Now that Hugging Face hosts it, users no longer need AWS credentials to access the data. This removes the need to maintain an extra secret and makes utilizing the data much more convenient.
More from Infra
- AI datacenters hit diseconomies of scale as inference shifts demand smaller — abhiadesai · 2026-07-22
- Nothing phone mockup turns a film joke into a modular design meme — ZeYanjie · 2026-07-22
- Actual Computer says its inference stack is tuned for Nvidia’s consumer Blackwell lineup — markjeffrey · 2026-07-22
- Ben Bajarin says CPU demand is still being badly underestimated — BenBajarin · 2026-07-22
- An energy model says the U.S. could run short of natural gas starting in 2028 — churchkey · 2026-07-22
- Devin adds e2b sandboxes for remote agent execution — badphilosopher · 2026-07-22