Homie: an offline Mac notch voice assistant running Qwen 3.5 4B locally
KungFuSaifooo · reddit · 2026-11-22
A developer released homie (MIT, open source), a voice assistant living in the Mac notch that runs fully offline.
- Stack: Qwen 3.5 4B (Q4KM) via llama.cpp on Metal, Kokoro-82M on the Neural Engine, Apple on-device STT;
- Key design: since the 4B model only writes 15–30 tok/s, everyday commands skip the LLM entirely — 600 lines of pattern matching handle them (214/214, zero wrong fires), with prompt state cached to disk;
- Performance: 0.43s p50 to first sound on commands, 1.17s on LLM turns, on an M5 16GB machine; an untested 2B tier targets 8GB Macs.
The README documents failed experiments too: spec decoding slower on this hybrid model, MLX at 34 vs 31 tok/s, quantization bake-offs — useful reference for local deployment tuning.
More from Infra
- AMD and PyTorch ship a playbook to run GPT-OSS-20B inference locally with ROCm — PyTorch · 2026-11-22
- Microsoft's full Outlook outage PIR confirms AI tooling wasn't to blame — aronchick · 2026-11-22
- User reports 400+ tok/s with Flare inference engine — capetorch · 2026-11-22
- Huawei Ascend-950DT quoted at 3x 950PR price; yield and SuperPod interconnect may explain the gap — teortaxesTex · 2026-11-22
- AlphaChip and three TPU generations: a mine that digs itself is a throne — krishnan · 2026-11-22
- SK Hynix reportedly tops 70% yield on HBM4, starting mass production in Q3 — Beth_Kindig · 2026-11-22