Hugging Face launches Buckets for hosting massive 33TB files
NielsRogge · x · 2026-08-23
Hugging Face released a new repository type called 'Buckets', allowing users to share and host extremely large files, marking the fourth type alongside models, datasets, and Spaces.
Key Features:
- Optimized for large files: Powered by Xet technology, it provides S3-like object storage optimized for AI workloads.
- Non-Git based: Unlike traditional Git repos, Buckets are remote object storage containers that do not track file history, making them suitable for storing training checkpoints, logs, intermediate artifacts, or large file collections that don't need version control.
- Deduplication & Speed: Supports content-addressable deduplication storage.
Use Case:
- @rerundotio released xdofai's ABC-130k bi-manual teleoperation dataset. The converted RRD files are hosted in a Bucket totaling 33TB, which users can visualize in Rerun or use directly in training workflows.
More from Infra
- NVIDIA releases Nemotron 3.5 Bolt 74B MoE that fits in 24GB VRAM at 2-bit — NerdyRodent · 2026-10-16
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16