Fitting a vector store into small hardware for local RAG
According-Floor5177 · reddit · 2026-07-20
The author asks how people fit a vector store into constrained hardware for local RAG.
They note that Weaviate’s HNSW index keeps a lot in RAM — roughly 6 GB for 1M vectors at 768 dimensions — which can consume an 8 GB machine before the model even loads.
They want practical answers from people running local RAG on small boxes:
- which vector store they use
- how many vectors fit before memory gets tight
- whether anyone uses disk-backed indexes in production, or just quantizes
- whether 768 dimensions is worth it locally, or smaller embeddings are the better trade-off
More from coding & agent
- An MCP server signs every AI agent tool call into a verifiable Merkle chain — Funky_Chicken_22 · 2026-07-22
- Claude Code skill uses 10 Markdown rules to make outputs ADHD-friendly — alex_verem · 2026-07-22
- A Firecracker-based platform says it can host 6,000 AI agents on one 256 GB server — maritime_sh · 2026-07-22
- A better path to agent autonomy is running waves, finding friction, and iterating — JnBrymn · 2026-07-22
- AI agent designers map the visual and tonal cues behind companionship products — Unlikely-Platform-47 · 2026-07-22
- Coding agents are heading toward an AI-writes, AI-reviews, human-approves workflow — aftahi_ai · 2026-07-22