DuckDB 2.0 Ships Native hf:// Extension with Xet Dedup Saving 80% Storage
lhoestq · x · 2026-10-22
Hugging Face engineer Quentin Lhoest released a native C++ DuckDB 2.0 extension for reading and writing hf:// URLs. Reads go through the Hub's /resolve/ endpoints with range requests and a shared read-ahead cache; writes use the Hub commit API with Xet-based deduplicated uploads. HF says its Xet-backed Buckets save 80% of storage globally. Install via INSTALL hf FROM hfrepo; LOAD hf FROM hfrepo; — available at hf.co/duckdb-extensions.
Related event: Hugging Face Ships DuckDB 2.0 Extension for SQL Access to Datasets(3 posts)→
More from Infra
- July 28 MCP release makes remote servers stateless, biggest revision since launch — bibryam · 2026-10-23
- MCP keynote: AAIF hits 305 members, Skills over MCP and Tasks/Events roadmap revealed — bibryam · 2026-10-23
- Only 8% of cloud vendors support OIDC — dev calls to kill long-lived API keys — cramforce · 2026-10-23
- Swift-1.5-Qwen3.8-27B ported to hipfire MQ4: 2.2x faster end-to-end than GGUF on a 7900 XT — Queasy_Asparagus69 · 2026-10-23
- Intel CTO: chip output can at best 2X while AI compute grows 3X per year — SammaVaco · 2026-10-23
- Engineer reverse-engineers projector TV's 10s wake lag to a cold-booting Linux board — banteg · 2026-10-23