Hugging Face ships Datasets 5.1 with Vortex, Harbor RL and five bio formats
lhoestq · x · 2026-10-05
Quentin Lhoest announced Datasets 5.1, largely carried by @bazarkhalili. Highlights: new formats including Vortex, Harbor for RL environments, and five biomolecule/protein formats; plus new streaming/iterable dataset features and a fix for shard counts when shuffling streaming datasets.
Related event: Hugging Face Releases Datasets 5.1 with Vortex and Biomolecule Formats(2 posts)→
More from Infra
- 691K H100-hours: community estimates compute behind from-scratch checkpoint trained on just 320 H100s — teortaxesTex · 2026-10-06
- Dev reports Clef-Flash runs fast locally even on memory-bandwidth-limited Jetson Orin — gregmushen · 2026-10-06
- Running LLMs Fully Client-Side: llama.cpp Compiled to WASM with WebGPU Proof of Concept — Numerous-Fan8138 · 2026-10-06
- Llama.wasm: llama.cpp Compiled to WASM with WebGPU Runs LLMs Fully in the Browser — Numerous-Fan8138 · 2026-10-06
- NVIDIA Dynamo lets coding agents point at self-hosted endpoints with native tracing — TheZachMueller · 2026-10-06
- Schmidhuber: compute gets 10x cheaper every 5 years, 100,000x in 25 years — SchmidhuberAI · 2026-10-06