llama.cpp tuning alone makes Qwen3-4B 1.34x faster on a 2019 phone, offline
hbouammar · x · 2026-10-15
A developer reports running Qwen3-4B fully offline on a 2019 phone and getting 1.34x faster responses (2.2x on long outputs) purely from llama.cpp settings — with identical outputs, no quantization or model changes needed. A useful datapoint for on-device LLM deployment.
More from Infra
- Eric Jang's three signals of AI overinvestment: compute repricing, open-source catch-up, robot cost halving — ericjang11 · 2026-10-15
- 2x AMD R9700 hits 176 tok/s decode and 6,450 tok/s prefill on Q4 Qwen3.8-Flash-Next — Athabasco · 2026-10-15
- Basalt hits 75 tok/s on a 5060 Ti: local LLMs on modest hardware — ironicstatistic · 2026-10-15
- Nvidia B300 residual value up 40% in 3 months as Kimi K3 fuels GPU demand — AccBalanced · 2026-10-15
- The world still spends more on alcohol ($1.7T) than AI infrastructure ($1.5T) — wgrathwohl · 2026-10-15
- Epic's Lore vs Anthropic's Scion: two post-Git, centralized VCS architectures compared — ssh4net · 2026-10-15