CPU-only inference on a $100 Celeron SBC shows 0.6B models are usable
tre7744 · reddit · 2026-07-24
CPU-only inference on a Celeron N5095 SBC shows that 0.6B models are usable, while 8B is not.
A Reddit user benchmarked six models on a Youyeetoo X1S single-board x86 machine with a Celeron N5095, 16GB RAM, and a 128GB NVMe, using Ollama in CPU-only mode. The board costs roughly $100–$130 depending on configuration.
Key results:
- Qwen3 0.6B averaged 6.788 tok/s and felt usable interactively
- an 8B model fit in memory but crawled at 0.924 tok/s
- the system stayed cool enough under a 15-minute all-core stress test, peaking at 77°C with no throttling
The author concludes that very small models can work well for classification, routing, and summarization on low-power x86 boxes, but 8B models hit a memory-bandwidth wall. Next up is a CPU vs Vulkan comparison using llama.cpp on the integrated GPU.
More from Infra
- Gemini CLI patch blocks credential leakage by forcing HTTPS for auth provider — amelidev · 2026-07-24
- AMD’s Ryzen AI Halo targets local AI apps with 128GB unified memory — ryanshrout · 2026-07-24
- A user wants an API layer that can start and stop local models on demand — minaminotenmangu · 2026-07-24
- Baseten and CapitalG set a demo night on owning the inference stack on August 4 — baseten · 2026-07-24
- AMD claims MI350P delivers 2–5x tokens per dollar in enterprise workloads — ryanshrout · 2026-07-24
- Databricks Genie runs as an MCP server inside LangGraph, then ships to Azure ML — Cautious-Meringue554 · 2026-07-24