'Fully local' RAG stacks can still quietly phone home
Competitive-Fee4307 · reddit · 2026-08-17
A post on privacy blind spots of local LLM setups: even if the model runs on your GPU, data can still leak at other hops:
- Embeddings: vector steps often default to a cloud API (e.g. OpenAI embeddings), so data leaves before the local LLM ever sees it;
- Telemetry: many loaders and frameworks report usage data unless explicitly disabled;
- Silent fallbacks: some stacks fall back to a cloud provider when the local path errors, without notice;
- Rerankers/OCR/parsing: easy to leave one component pointed at a hosted API.
The author asks the community whether anyone runs local stacks for real on regulated data, and whether they verify end-to-end via egress audits and pinning every component local.
More from Infra
- Musk says he wants 'maybe more than 10GW' of compute; Nvidia now counts SpaceX as a CSP and co-designs a space server — elonmusk · 2026-10-16
- Jensen Huang: Musk's AI buildout moved from K's to M's, now a CSP-tier NVIDIA customer — elonmusk · 2026-10-16
- EpochAI pegs GPT-6 training at up to 1.8e27 FLOP; scaling01 calls param estimates way too high — scaling01 · 2026-10-16
- Meta's Muse Spark Open scores 39 vs 48 for closed Spark 1.3 on AA Intelligence Index — EAccelerate_42 · 2026-10-16
- LLM spots useless 95GB-per-frame memcpy in ffmpeg dblur, cutting runtime from 2452ms to 49ms — IgorCarron · 2026-10-16
- DeepSeek-V4-Flash on 4x Radeon AI PRO R9700: the 'refused' 4-card config hits ~79 tok/s — brainchillzZ · 2026-10-16