SQLite is ~35% faster than the filesystem for 10KB blobs — here's why the syscall overhead adds up
arpit_bhayani · x · 2026-11-20
The author explains why SQLite beats raw files for small blobs: each file read pays open/read/close syscalls, directory lookups and permission checks — 100,000 times for 100,000 files — while SQLite keeps one file open and reads via B-tree lookups in the page cache. Filesystems allocate in 4 KiB blocks (a 10KB file costs 12 KiB), whereas SQLite packs records tightly into pages. Atomic updates of 50 files need temp files, fsyncs and renames with raw IO; SQLite gives journal-based atomic commits out of the box. Trade-offs: the advantage fades for larger blobs, single-writer is a real limit, and one huge DB file is harder to inspect.
More from Infra
- Local inference users want 512GB RAM and 1M context, says current boxes fall short — AIFlow_ML · 2026-11-20
- BitDecoding-FP4: native FP4 KV cache decoding via algorithm-system co-design — kalyan_kpl · 2026-11-20
- Prefill beats decode: SGLang wins agent runs on DGX Spark despite slower tok/s — AIFlow_ML · 2026-11-20
- Nvidia supplier commitments jump from $87B to $203B as forecasts miss low — tengyanAI · 2026-11-20
- Qwen3.8-Flash-Next on a single Intel Arc B70: 2.5K prefill, 47 tok/s — WizardlyBump17 · 2026-11-20
- NVIDIA's NarrowSpec: Halving Draft Hidden Width Still Speeds Up Speculative Decoding 1.44x — nvidia · 2026-11-20