Building a Local RAG Stack on Mac Studio: Read/Write Separation and MCP
Joules_Jokes_Leks · reddit · 2026-08-01
The author shares a completely local enterprise Wiki read/write system running on a Mac Pro M2 Ultra (128GB). The system uses hybrid retrieval and is served via the MCP protocol.
- Read Design: Uses Qwen3-Embedding-4B and BM25 for hybrid retrieval, fused with RRF, and reranked by bge-reranker-v2-m3. A strict score threshold refuses to return low-confidence answers, preventing junk responses.
- Write Design: An orchestrator feeds retrieved context to a local Qwen3.6-27B for drafting. The model only writes based on evidence, applying templates, citing sources, and marking gaps before human review. Low scores trigger a refusal, eliminating hallucinations.
- The Loop: Refusal logs double as a backlog for missing docs. Nightly delta syncs keep the index fresh, and published drafts become future sources.
In an overnight run, the system processed 106 stub pages into 105 grounded drafts and 1 correct refusal, taking 3-5 minutes per page.
More from coding & agent
- Replit launches Design feature with templates and ambient intelligence — amasad · 2026-08-01
- LOLM: A Hybrid Transformer-SSM Agent Architecture Open-Sourced — OGMYT · 2026-08-01
- RCE Vulnerability Disclosed Across Multiple Official MCP SDKs — SelectionBitter6821 · 2026-08-01
- Supabase Launches AI Coding Agent Benchmark, Endorsed by Paul Graham — lavanyaai · 2026-08-01
- Simon Willison Releases smevals: A Lightweight Eval Suite for LLMs — Simon Willison · 2026-08-01
- Seeking Help: Moving from Cloud to Local Self-Hosted Video Puppeteering Pipeline — PingO_Oatmountain · 2026-08-01