Hugging Face Ships DuckDB 2.0 Extension: Query and Write hf:// Datasets in SQL
lhoestq · x · 2026-10-22
Hugging Face engineer lhoestq announced an official hf extension for DuckDB 2.0, available on day one. After installing it, you can read and write HF datasets, models and Spaces directly in SQL:
sql
SELECT FROM 'hf://datasets/fka/prompts.chat/prompts.csv';
COPY (SELECT ...) TO 'hf://datasets/me/demo/out.parquet';
- hfls lists bucket contents like Common Crawl; partitioned Parquet writes are supported.
- Storage is optimized via Xet, claimed to be among the most storage-efficient cloud options.
- Auth via hf auth login or CREATE SECRET with a token.
Open-sourced on GitHub as duckdb-hf-extension.
Related event: Hugging Face Ships DuckDB 2.0 Extension for SQL Access to Datasets(3 posts)→
More from Infra
- IFM open-sources RL360: 8,192 concurrent sandboxes and ~1.3M rollouts for agentic RL — rupspace · 2026-10-23
- SpaceX seeks access to 80,000 US cell tower sites; Polymarket puts carrier odds at 29% — Polymarket · 2026-10-23
- Modular open-sources MAX framework to free AI compute from single-vendor lock-in — clattner_llvm · 2026-10-23
- Claude Models on Google Cloud (Anthropic managed) is now generally available — claudeai · 2026-10-23
- Scaling GLM-5.3-Flash training to 1M token context, 64x over Megatron baseline — AndrewLBeam · 2026-10-23
- Perplexity's Portable Computer lands on AMD Ryzen AI Max, running local 27B model — AravSrinivas · 2026-10-23