What do people actually use offline LLMs on iPhone for?
James333i · reddit · 2026-07-26
A long discussion about mobile offline LLMs on iPhone asks what people actually use them for.
The poster says they’ve spent the past year testing open-source MLX and GGUF models on iPhone hardware and found a few practical patterns:
- Apple Foundation models are limited, but useful for tool calling, fast summarization, and classification before handing work to stronger models.
- Models from 0.5B to 8B can work on higher-end devices.
- They’ve used them for:
- web search and URL scraping
- summarization
- research
- analyzing local photos, videos, and documents
- basic on-the-go coding help
- With continuous compaction, they claim to keep chats effectively unbounded despite 8k–16k token context limits.
- They also suggest private, offline personal chat as another promising use case.
The main question is whether mobile is still underexplored compared with laptops and desktops.
More from Infra
- A100 rental prices still rising in 2026 per a16z data, Jevons paradox hits compute — VraserX · 2026-10-27
- Anthropic S-1 reveals $800B+ in off-books future compute commitments — GaryMarcus · 2026-10-27
- ~42% of Anthropic's ~11GW contracted compute is NVIDIA GPUs — ns123abc · 2026-10-27
- EdgeRock-4B holds KV cache at 72 MiB across 256K context, 99.1% smaller than Qwen3.5-4B — alifcoder · 2026-10-27
- RTX 5080 price jumps from $1658 to $2318 in ten days on Amazon — Frizzy-MacDrizzle · 2026-10-27
- ComfyUI on 12GB VRAM: measured peaks show offloading turns 24s into 11 minutes — Impossible-Mail-4619 · 2026-10-27