Datasets 5.1 follow-up: new streaming features and a shuffle shard-count fix
lhoestq · x · 2026-10-05
A follow-up on Datasets 5.1: the release also brings new features for streaming/iterable datasets and fixes the number of shards when shuffling a streaming dataset, thanks to @yuchenwang03.
Related event: Hugging Face Releases Datasets 5.1 with Vortex and Biomolecule Formats(2 posts)→
More from Infra
- Bank of America warns 'easy money' from the AI spending boom may be ending — Polymarket · 2026-10-06
- VC doubles down on inference as software's most important market, surpassing databases — buckymoore · 2026-10-06
- $3,500 Blackwell Personal AI PC: RTX PRO 4000 Runs Qwen Next at 50-70 tok/s — Jackyhuang · 2026-10-06
- Bought an RTX 5060 for local LLMs — complex tasks scored 2/10 vs 9/10 in the cloud — Tricky-Brother-7 · 2026-10-06
- Reka CEO: we have the training stack and data, just not the compute — seeking partners — RekaAILabs · 2026-10-06
- 64GB Halo Strix Runs 27B Locally: Should This User Switch to Qwen Flash? — HyenaUpbeat · 2026-10-06