huggingface_hub 2.3.0 cuts 1M-file listings to 20s, adds live Bucket following
lhoestq · x · 2026-10-27
Hugging Face engineer Quentin Lhoest highlights the v2.3.0 release of huggingfacehub and its upgrades for HF Buckets, already in heavy internal use:
- Much faster listings: enumerating 1M files now takes 20 seconds instead of minutes
- Live Bucket following (livefollow=True): HfFileSystem opens a background connection per followed Bucket to receive file-change events, so files added or deleted by other processes show up in ls() without a manual invalidatecache()
The option also works through fsspec (fsspec.filesystem("hf", livefollow=True)) and via storageoptions in libraries like pandas and pyarrow. It's off by default since each followed Bucket holds an open connection.
More from coding & agent
- SF 'Code Engineering' event picks up where 1,000-person harness engineering meetup left off — LangChain · 2026-10-27
- Prompt turns Claude Code into an iPhone backup forensics tool exposing 30-day 'deleted' photo copies — alex_verem · 2026-10-27
- Prompt turns Claude Code into a forensics tool finding deleted iPhone photos in backups — alex_verem · 2026-10-27
- Phones won't be mobile IDEs but control planes for agent fleets, argues Yuchen Jin — Yuchenj_UW · 2026-10-27
- FrontierCS 2 launches on Kaggle: 48 tasks make agents beat published CS papers — MeganRisdal · 2026-10-27
- Stop remaking old games with AI: a call for more ambitious agentic software — rschu · 2026-10-27