s3collections: Durable Go data structures backed directly by S3-compatible storage
andersonbcdefg · x · 2026-08-24
Engineer DamianB released s3collections, a library providing durable Go data structures and primitives (like LRU caches, queues, and trees) backed by S3-compatible storage. The project uses a hybrid storage model: small transactional metadata is stored in SlateDB on S3, while larger data bodies remain as plain blobs. Reflecting on 7 years of engineering practice, the author suggests using S3 is a more practical approach for distributed systems substrates compared to implementing complex consensus algorithms like Raft from scratch. It offers a simple API to share state across a fleet without a traditional database.
More from Infra
- Offloading MoE models to RAM causes slow prefill speeds — former_farmer · 2026-08-24
- Etched Raises $1B Led by Jane Street to Validate Architecture-Agnostic AI Chips — TheTuringPost · 2026-08-24
- ConvRot Quant joins llama-cpp: Q6 accuracy nears Q8 quality — giveen · 2026-08-24
- LifeOS: A Local, Voice-Driven Personal Organizer — Extension-Bid-639 · 2026-08-24
- Hyperscalers: Choosing Between HDD and SSD Based on Space and Cost — generativist · 2026-08-24
- Samsung shows new HBM cooling solution, hints at die performance variance — BenBajarin · 2026-08-24