DuckDB extension stores full native databases in S3 with nothing else to run

josh_wills · x · 2026-10-13

• The community extension duckdbobjectstorage, built on SlateDB and Apache OpenDAL, lets you ATTACH 'duckdbobjfs://analytics.db' and keep a native DuckDB database (indexes, constraints, WAL, checkpoints) directly in S3.
• No catalog service, Postgres, coordinator, or compactor needed—all metadata, even writer fencing, lives in object storage. One writer, many readers, never inconsistent, making it the storage layer for the author's distributed engine Duckherder.
• Reads are within 10% of native local DuckDB; cold S3 reads are 1.2x native over HTTPFS. Tested with DuckDB's own test suite.

Original post →

More from Infra

• Microsoft, AWS, GCP may hit $100-120B AI revenue in 2026, doubling YoY — BenBajarin · 2026-10-14
• Prime Intellect, Valued at $1B, Acquires Moondream to Speed Up Open Model Inference — himanshustwts · 2026-10-14
• Applied Materials CEO: "We Manufacture Miracles All the Time" — BenBajarin · 2026-10-14
• Cloudflare D1 Scales to 5.5 Million Databases: Inside the Control Plane Architecture — Cloudflare Blog · 2026-10-14
• Gavin Baker: open-weight tokens will dominate, B200 clusters still yield 37% IRR — GavinSBaker · 2026-10-14
• AMD's Lisa Su sees strong AI inference and CPU demand at OCP summit — firstadopter · 2026-10-14

Infra channel →