apple-llm: Node/Python wrapper for the free local LLM built into Apple Silicon Macs
light_2earth · reddit · 2026-09-28
Apple Silicon Macs running macOS 26+ ship with a small built-in LLM — no download, no API key, nothing leaves the machine. The author hit real pitfalls wiring it up (repetition at temperature 0, 2s calls turning into 20s, 17s calls until keeping one process alive) and packaged the fixes into apple-llm (npm / pip install apple-llm).
Good for: structured extraction from messy text (emails→tickets, receipts→expenses, schema-matching JSON), tagging/summarising/rewriting, private data, and tools shared with Mac users who need no model or key. Bad at: coding and long reasoning. An optional cloud mode uses Apple's larger server model, but prompts leave the device and carry usage limits. GitHub: github.com/jagdishpal02000/apple-llm
More from coding & agent
- Agent failed for weeks on a problem StackOverflow answered instantly — itsOmSarraf_ · 2026-09-28
- Harness-Zero: PKU, Google and HKUST Distill Agent Harnesses into Model Weights — AxSaucedo · 2026-09-28
- PromptSentry: open-source 3-layer proxy blocks prompt injections in under 1ms with local DLP scrubbing — Ok-Negotiation342 · 2026-09-28
- ProgramBench Called an Excellent, Far-from-Saturated Eval for Multi-Agent Coding Systems — jyangballin · 2026-09-28
- Prompt engineering isn't dead, says dev: it just shifted from wording tricks to defining tasks clearly — Luvena21 · 2026-09-28
- What agent sellers miss: you can't outsource accountability — OgundipeOre · 2026-09-28