llama.cpp gotcha: -ngl -1 means auto, not all — switching fixed CPU usage and 60→76 t/s
randomjapaneselearn · reddit · 2026-11-13
On a 3090 running Swift-1.5-Qwen3.8-27B Q4KM, the poster hit high CPU usage: with -ngl -1, even at 131K context (below the auto-suggested 150K), logs showed inconsistent offloading (64-66 of 66 layers, once 0). Changing to -ngl all reliably offloaded 66/66 layers, eliminated CPU usage, and raised generation speed from 60 to 76 t/s. Conclusion: -ngl -1 actually means "auto" while -2 means "all", and the --help text calling it a max limit is misleading. If a model should fit in VRAM but CPU usage spikes, just set all.
More from Infra
- Microsoft outage caused by unresponsive servers during change validation — Kyrannio · 2026-11-13
- 'Compute won't escape overbuild': trader John Arnold on the coming capacity glut — McDonaghMatthew · 2026-11-13
- AWS tests four Bedrock settings that cut inference cost: cache TTL, Flex tier, effort, maxTokens — AWS ML Blog · 2026-11-13
- Cloudflare Workers rate limiting now supports delta-based usage in limit() calls — ritakozlov · 2026-11-13
- RL engineering debate: decoupled rollout vs colocated vLLM, staleness is not a rounding error — willcb · 2026-11-13
- WakeGPU: open-source desktop app to run remote Colab GPUs with auto idle shutdown — Far-Ad9512 · 2026-11-13