Ollama beats llama.cpp by 16-35% on Celeron N5095 CPU; Vulkan cuts temps 27C but breaks on big models
tre7744 · reddit · 2026-09-02
A rigorous matched benchmark of Ollama 0.32.1 vs llama.cpp (commit 9a286ac) on a low-power Celeron N5095 board, using identical prompts, Q4KM files, 4,096-token context, four threads, and 96 generated tokens.
Findings
- Ollama's CPU runner was faster across the board: 16.1%–34.9% higher generation throughput (e.g. Qwen3 4B: 2.002 vs 1.484 tok/s).
- Full Vulkan on the Intel iGPU sped prompt processing 3.31–3.50x, improved generation 16%, and dropped package peaks from 81C to 57C.
- Larger models broke under Vulkan: i915 resets on Qwen3 4B, GPU hangs on Phi-4 Mini and Qwen3 8B, DeviceLostError on Gemma 3.
- Zero GPU layers doesn't mean pure CPU — llama.cpp still offloads host ops to the iGPU.
- MTP was slower than disabled at every depth on Qwen3.5 0.8B/2B and Gemma 4 E2B.
Bottom line: keep Ollama for CPU, restrict Vulkan to Qwen3 0.6B/1.7B, leave MTP off. Full write-up, scripts and failure timeline are public.
More from Infra
- Tencent open-sources CubeSandbox v0.7.0, keeping thousands of agents alive through node failures — SucceededMind · 2026-09-03
- Magnitude open-source inference server swaps coding agents to free local models automatically — nickbaumann_ · 2026-09-03
- MiniMax H3 video generation runs fully local on a single RTX 5060 Ti 16GB — apoke890 · 2026-09-03
- With 98% Cache Hits in Coding, 400-600 t/s Prefill Already Hits Diminishing Returns — nomorebuttsplz · 2026-09-03
- Dell COO: inference tokens to grow 87x to 3,600 quadrillion by 2030 — Beth_Kindig · 2026-09-03
- Beating hipBLASLt on a gaming GPU: a GEMM optimization deep dive — Moist_Weird_42067 · 2026-09-03