Offline village AI: $5,000 budget to build a local LLM machine for basic Q&A, seeking GPU advice
Potential_Low_1183 · reddit · 2026-09-07
A user's grandfather lives in a remote Asian village with no internet but working phone service. After falling in love with ChatGPT, he wants a local LLM machine for basic knowledge and light Q&A, with a budget of about $5,000—villagers may also use it.
The poster runs DeepSeek V4 Flash on two RTX 6000 Pro Blackwells and is considering AMD AI Pro 9700 (possibly two) with a quantized 31B Gemma. The goal is reliable knowledge base Q&A, not frontier coding, and they're seeking GPU recommendations.
More from Infra
- 928-Star Wiki Details Running Qwen3.5-397B and Kimi-K2.5 on NVLink-Free PCIe RTX 6000 — TheZachMueller · 2026-09-07
- Prediction: compute is moving from rack-scale to datacenter-scale as bottlenecks shift outward — AccBalanced · 2026-09-07
- AI meme: turning off the tap while brushing teeth to "conserve water for the datacenter buildout" — EigenGender · 2026-09-07
- AMD MI355X beats NVIDIA B300 on tokens-per-dollar in AgentX — AccBalanced · 2026-09-07
- RandKV ships as pip-installable random KV-cache eviction for Transformers, reports honest negative perf results — atease01 · 2026-09-07
- Custom llama.cpp Branch Adds Expert Expansion for MoE Models — Specific-Tax-6700 · 2026-09-07