Run your own AI model on your laptop this weekend: a privacy-first local setup guide
alex_verem · x · 2026-11-11
The author makes the privacy case for local models: everything you tell ChatGPT—symptoms, relationship fights, debts, contracts, months of fine-tuned style—lives on someone else's servers. Sam Altman himself said on a podcast in July 2025 that ChatGPT conversations lack the legal privilege therapists and lawyers have, and OpenAI could be compelled to hand them over in lawsuits.
Key points from the guide (with charts):
- Memory is the constraint: a model must fit in RAM to run at full speed; on 16GB machines an 8GB Gemma 4 12B fits, while the 18GB Qwen 3.8 27B spills to disk and slows generation.
- Pick by RAM: 8GB → Qwen 3.5 4B (3.3–4GB download); 16GB → Gemma 4 12B or Qwen 3.5 9B; 32GB+ → Qwen 3.8 27B or Gemma 4 26B. All four are Apache 2.0 licensed.
- 20-minute, 7-step setup: check RAM → pick a fitting model → install LM Studio or Ollama → turn off Wi-Fi to verify it's truly local → fix slowness with a smaller model/shorter chats → back up the model folder → save a system prompt as a Preset. Zero accounts, subscriptions, or internet needed afterward.
More from Infra
- Huawei's Kirin 9050 NPU claims offline 30B MoE on smartphones, +140% perf — AIFlow_ML · 2026-11-11
- Stratechery: Intel vs AMD's diverging earnings and Intel's need for an old-new path — Stratechery · 2026-11-11
- a16z Podcast: Mike Volpi on What the 1990s Internet Boom Teaches Us About the AI Buildout — a16z Podcast · 2026-11-11
- Alibaba files to list chip-design unit T-Head in Hong Kong to fund AI push — pstAsiatech · 2026-11-11
- AWS CEO expects to be short on compute 'for the next couple of years' as demand outstrips supply — firstadopter · 2026-11-11
- Natural language search over autonomous driving data with Cosmos-Embed1 + OpenSearch — 4310sy · 2026-11-11