Building a Private RAG System for 50 Users: A Mac Mini Cluster Proposal
rogo725 · reddit · 2026-08-07
A user shared a proposal for building a fully private, self-hosted RAG system without cloud APIs for a distributed team of 50 users.
- Hardware Setup: The plan involves a small cluster of Mac Mini Pros (M4 Pro, 48GB RAM) with a load balancer and VPN access, plus a standby node for failover.
- Model Choice: Aiming to run a 30B-35B class open-weight model across independent nodes, prioritizing simplicity and reliable concurrent access over pooling resources for a massive model.
- Core Needs: The setup must handle real concurrent usage, redundancy, and efficient document ingestion as the library grows.
The user is seeking feedback from anyone with practical experience in similar self-hosted infrastructure.
More from Infra
- NVIDIA Unveils Vera Rubin NVL72 Compute Tray: 100% Automated, No Cables or Fans — MickeySteamboat · 2026-08-07
- Cloudflare Launches Kitesurf: A Headless Browser Built for AI Agents — dinasaur_404 · 2026-08-07
- Trillion-Dollar GPU Bets and AGI Delays: Is the AI Boom Sustainable? — Positive_County_3493 · 2026-08-07
- a16z on Open Weight Models: Cost and Control Are the Core Advantages — FinanceYF5 · 2026-08-07
- SK Hynix 2Q26 OPe Forecast Misses Consensus Due to High HBM Mix and LTA Effects — zephyr_z9 · 2026-08-07
- Google TPU v7s Are Not Sold Cheap, Hardware Costs Remain High — zephyr_z9 · 2026-08-07