What do people actually use offline LLMs on iPhone for?
James333i · reddit · 2026-07-26
A long discussion about mobile offline LLMs on iPhone asks what people actually use them for.
The poster says they’ve spent the past year testing open-source MLX and GGUF models on iPhone hardware and found a few practical patterns:
- Apple Foundation models are limited, but useful for tool calling, fast summarization, and classification before handing work to stronger models.
- Models from 0.5B to 8B can work on higher-end devices.
- They’ve used them for:
- web search and URL scraping
- summarization
- research
- analyzing local photos, videos, and documents
- basic on-the-go coding help
- With continuous compaction, they claim to keep chats effectively unbounded despite 8k–16k token context limits.
- They also suggest private, offline personal chat as another promising use case.
The main question is whether mobile is still underexplored compared with laptops and desktops.
More from Infra
- 12 KV Cache Reduction Techniques Every AI Engineer Should Understand, Explained — blaizedsouza · 2026-09-11
- The shadow GPU capacity market is formalizing, with Meta selling excess compute to outside buyers — DavidLinthicum · 2026-09-11
- Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs — bookwormengr · 2026-09-11
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11