LokalBot: Open-Source Mac App Orchestrates 5 Local Models for Meetings
stevyhacker · reddit · 2026-11-10
After 6 months of work, the author open-sourced LokalBot (GPLv3), a Mac app chaining five local models — Qwen3-ASR 1.7B, Nemotron 3, Qwen3.5 4B, Gemma 4 E2B, EmbeddingGemma 2 (9.6 GB total) — for meeting recording, transcription, notes, semantic search, and autocomplete, fully on-device.
Scheduling: models load on demand and unload after idle timers (120s for speech models, 10-min leases for servers); autocomplete stays in-process until macOS memory pressure; an LRU broker caps loaded weights at half of RAM. Key gotcha: MLX pools freed GPU buffers — the app held 4.3 GB idle until the pool was cleared on unload.
Benchmarks (M4 Max 48GB): 26-min meeting notes in 33s warm / 60s cold at 85 tok/s; autocomplete 56ms median / 102ms p95; query embedding 9ms; 14.6% DER at 500× real time. Only tested on one machine; Windows/Linux in alpha. The narration voice is ElevenLabs-cloned — the only part not run locally.
More from Infra
- Daytona CEO: Model inference is just a small slice of the AI agent infrastructure opportunity — mattturck · 2026-11-10
- Coherent signs $20B in laser orders shipping from 2028 for AI CPO optics — demian_ai · 2026-11-10
- Invest in engineering, not agent factories: Geoffrey Huntley says Nix and fast builds get uplifted by model drops — sloppenheimer · 2026-11-10
- fast_float benchmarks: parser hints double throughput to 355M numbers/sec — lemire · 2026-11-10
- Anthropic retracts IPO filing after sending bankers wrong price range; Google, Nvidia, Microsoft take $26B private slice — GaryMarcus · 2026-11-10
- Taiwan supply chain: Nvidia CEO controversy won't derail AI demand long-term — nordicinst · 2026-11-10