Artificial Analysis launches benchmarks for small models on mobile phones
ArtificialAnlys · x · 2026-08-25
Artificial Analysis, in partnership with Liquid AI, has launched intelligence and inference benchmarks for small models on mobile phones. The intelligence benchmark uses five evaluations tailored for function calling, knowledge, and reasoning tasks. The inference benchmark measures speed, latency, and memory on real devices. The article highlights that benchmarking small models is difficult due to the clustering of scores on traditional frontier benchmarks and the immaturity of mobile runtimes. Tests use 4-bit or smaller quantized builds.
Related event: Artificial Analysis and Liquid AI Launch On-Device Small Model Benchmark(9 posts)→
More from Infra
- NVIDIA releases Nemotron 3.5 Bolt 74B MoE that fits in 24GB VRAM at 2-bit — NerdyRodent · 2026-10-16
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16