Can a $100 MacBook Pro M2 24GB serve local LLMs for Home Assistant?
Ejo2001 · reddit · 2026-10-22
The poster got a used MacBook Pro M2 (24GB unified memory, 1TB) for about $100 and wants to run llama.cpp or vLLM on it as a lightweight local AI server for Home Assistant and family, hoping to cut the electricity bill of their existing rig (Threadripper + 4× RTX 5060 Ti 16GB).
Concerns after research:
- M-series can likely serve 9B-14B models, but prompt processing is slow
- Worried hallucinations make it unreliable for Home Assistant voice control
- An old thread recommended Qwen3:30B; unclear if that's still needed
Hands-on: gemma4:e4b was okay but fumbles multi-step instructions — "turn off all lights except light 3" turns everything off and then claims light 3 is still on. Asking the community for model recommendations and optimizations.
More from Infra
- Dev shows off self-built infra stack — HTTP clients, attention kernels, model routers — to quit dependencies — wavefnx · 2026-10-22
- Tune your threshold before adding an LLM judge: optimal escalation rate was 0% — Reasonable_Royal_621 · 2026-10-22
- Where your $20 Claude subscription goes: an interactive AI value-chain explainer — aishashok14 · 2026-10-22
- llama.cpp WebGPU adds Q2_0 support, Ternary Bonsai prefill jumps ~50x to 2000 t/s — pmttyji · 2026-10-22
- CVE-2026-111943: Compromised intermediate certificate authority discovered — keks24 · 2026-10-22
- AI's hidden storage bill: an 8-second video leaves 500MB behind — MIT Tech Review AI · 2026-10-22