AUR llama.cpp-cuda removal caused 10x slowdown; manual rebuild restored 1800 t/s prefill
MrHall · reddit · 2026-09-08
- A Reddit user found that after the AUR llama.cpp-cuda package was deleted, switching to the recommended ggml-cuda + llama.cpp combo (v3 optimized CachyOS builds) dropped performance to about 10% of previous speed.
- Symptom: the model loads fully onto the GPU, but processing appears to happen on the CPU.
- By recovering the old PKGBUILD and using its build script, with latest CUDA optimizations (for an RTX 5070) and CPU optimizations, prefill throughput recovered from 150 t/s to 1800 t/s.
- The poster asks why official binaries lag so far behind a custom build and complains that llama.cpp updates too often to maintain manual builds daily.
More from Infra
- Engineer with thousands of fabricated devices pours cold water on photonic crystal modulator yields — jwt0625 · 2026-10-26
- Anthropic S-1 reveals 48 cents of compute cost per dollar of Q2 revenue — nmasc_ · 2026-10-26
- Persistent AI joins UR network to route private AI inference with encrypted, IP-hidden traffic — 0xSammy · 2026-10-26
- Anthropic's disclosed compute commitments: $114.5B Google, $161.2B Broadcom top the list — BenBajarin · 2026-10-26
- SpaceX strikes $40B deal with Apollo Global to lease AI chips and data center hardware — Polymarket · 2026-10-26
- SpaceX AI revenue up 830%, Claude designs 318K antivenom proteins in 64 hours — PeterDiamandis · 2026-10-26