Mobile Benchmark: LFM2.5 Leads Efficiency on iPhone 17 Pro
ArtificialAnlys · x · 2026-08-25
Artificial Analysis, in partnership with Liquid AI, released intelligence and inference benchmarks for small models on mobile devices like the iPhone 17 Pro and Galaxy S26 Ultra, limiting context to 16K to simulate mobile memory constraints.
Key Findings (16K Context):
- Top Tier: Nanbeige4.2-3B and LFM2.5-2.6B tie for the top average score of 63.
- Efficiency: LFM2.5-2.6B is more efficient, processing a 1,024-token prompt in 8.0s using 2.3 GB memory on an iPhone 17 Pro, compared to 21.4s and 4.0 GB for Nanbeige4.2-3B.
- Context Impact: With a 64K context window, Ling 3.0 Tiny takes the lead with a score of 66.
The inference benchmarking app "Pipette" is available for free on iOS and Android.
Related event: Artificial Analysis and Liquid AI Launch On-Device Small Model Benchmark(9 posts)→
More from Models
- Unredacted Anthropic filing exposes internal data-quality debate months before Opus 3 cutoff — zacharynado · 2026-10-15
- Google says teams across the company are using Gemini 4 Argon on its hardest engineering problems — CurieuxExplorer · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Leak: Moonshot's Kimi flagship 'x4' done with pre-training, K3.1 in internal testing — lyraxana · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15