On-device small model selection: Gemma 4 and Qwen3.5 in 2026
lmoroney · x · 2026-08-22
The article analyzes small models (2B-12B parameters) suitable for on-device deployment on phones and laptops. It prioritizes Google's Gemma 4 (Apache 2.0) and Alibaba's Qwen3.5 (Apache 2.0), detailing Gemma 4's size range, performance metrics on Raspberry Pi 5, and the 12B multimodal variant. It compares licensing for SmolLM3 and Llama 4, emphasizing the importance of checking licenses and memory bandwidth for local deployment, and advocates for a hybrid architecture where local models handle simple queries and cloud models handle complex ones.
Related event: Gemma 4 and Qwen3.5 Top Picks for On-Device Models in 2026(2 posts)→
More from Infra
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16
- Samsung cuts Q4 phone output as Nittobo shifts E-glass to AI-chip T-glass — zephyr_z9 · 2026-10-16