Pocket AI Lab: open-source iOS app runs LLMs fully on-device with three backends
Ammoryyy · reddit · 2026-09-10
A developer released Pocket AI Lab, a free, MIT-licensed iOS app that runs LLMs entirely on-device — no cloud, no account, nothing leaves the phone.
Key details:
- Three backends side by side: MLX, llama.cpp and Core ML, picked per model. Paste a Hugging Face link and the app detects format, lists quants and predicts fit before downloading; you can also import .gguf files or pick from a curated list that updates without an app release.
- iOS memory engineering: third-party apps face a hard jetsam limit of roughly half the RAM. llama.cpp mmaps GGUF weights as file-backed pages that jetsam doesn't count — 8GB usable on an 11GB phone — while MLX and Core ML copy weights into dirty buffers and get 4.5GB. Same phone, same model: one backend runs it, another refuses. The app computes per-device budgets and refuses oversized loads instead of dying mid-run.
- On-device benchmarking: load time, prompt processing, tokens/sec with standard deviation, peak memory, thermal state, plus machine-checkable quality tasks. A build test downloads and benchmarks every catalog model your device can run, producing one report.
First-release bugs (two backends missing from the shipped build, long-prompt crashes, wrong memory budgets) are fixed. Code on GitHub, app on the App Store.
More from Infra
- Anthropic S-1 reveals 48 cents of compute cost per dollar of Q2 revenue — nmasc_ · 2026-10-26
- Persistent AI joins UR network to route private AI inference with encrypted, IP-hidden traffic — 0xSammy · 2026-10-26
- Anthropic's disclosed compute commitments: $114.5B Google, $161.2B Broadcom top the list — BenBajarin · 2026-10-26
- SpaceX strikes $40B deal with Apollo Global to lease AI chips and data center hardware — Polymarket · 2026-10-26
- SpaceX AI revenue up 830%, Claude designs 318K antivenom proteins in 64 hours — PeterDiamandis · 2026-10-26
- Meta plans up to $95 billion stock sale to fund AI buildout — The Decoder · 2026-10-26