llama.cpp Local Inference Debugging: Sudden 50% TPS Drop
campaigner_ · reddit · 2026-08-02
A developer encountered a bizarre issue where their local inference speed (TPS) using llama.cpp suddenly halved on an RTX 3070 + i7-11700 setup.
The Issue:
Previously, running Qwen and Gemma models maintained a stable 26-30 t/s. Recently, the speed dropped to 12-13 t/s at startup, and reducing the context size from 131k to 32k didn't help.
The author shares their full startup parameters (including --cpu-moe, --no-mmap, --chat-template-kwargs, etc.) to ask the community for help in troubleshooting whether the bottleneck is due to parameter conflicts or underlying updates.
Related event: Windows Reset Fixes Halved Local LLM Inference Speeds on RTX 3070(2 posts)→
More from Infra
- Ibiden’s AI substrate pricing surge sets up a clean earnings asymmetry — tengyanAI · 2026-08-04
- Big Tech’s OpenAI and Anthropic stakes are inflating reported earnings — Kr00ney · 2026-08-04
- Menlo Ventures says AI has entered phase 2, with infrastructure as the real opportunity — mmurph · 2026-08-04
- Podcast says AI CapEx, compute crunch, and debt-financed data centers are squeezing semis — BenBajarin · 2026-08-04
- MiniMax H3 open weights run 32 minutes down to 7.2 minutes on an L40S — ashishsanu · 2026-08-04
- Fluidstack takes its AI infrastructure dinner series to Austin and keeps hiring — MxMnr · 2026-08-04