Mobile Benchmark: LFM2.5 Leads Efficiency on iPhone 17 Pro
ArtificialAnlys · x · 2026-08-25
Artificial Analysis, in partnership with Liquid AI, released intelligence and inference benchmarks for small models on mobile devices like the iPhone 17 Pro and Galaxy S26 Ultra, limiting context to 16K to simulate mobile memory constraints.
Key Findings (16K Context):
- Top Tier: Nanbeige4.2-3B and LFM2.5-2.6B tie for the top average score of 63.
- Efficiency: LFM2.5-2.6B is more efficient, processing a 1,024-token prompt in 8.0s using 2.3 GB memory on an iPhone 17 Pro, compared to 21.4s and 4.0 GB for Nanbeige4.2-3B.
- Context Impact: With a 64K context window, Ling 3.0 Tiny takes the lead with a score of 66.
The inference benchmarking app "Pipette" is available for free on iOS and Android.
Related event: Artificial Analysis and Liquid AI Launch On-Device Small Model Benchmarks(8 posts)→
More from Models
- Venice.ai reportedly integrates Gemma 4 Uncensored model — EnigmaFund · 2026-08-25
- Debate over DeepSeek V4 code design, claims it is not distilled from Opus — MaziyarPanahi · 2026-08-25
- GLM-5.3 released with 1M-token context window — thione · 2026-08-25
- OpenAI has reduced GPT-5.6 Sol API pricing until at least November 21, in a move to encourage more API usage of the model. — petrusenko_max · 2026-08-25
- Free Tiny AI Qwen2.5-72B Delivers Surprisingly High Performance — Two Minute Papers · 2026-08-25
- Leaked Wandb Logs Hint at Potential 'Most Significant' AI Model Drop This Year — wandb · 2026-08-25