llama.cpp Fixes Redundant Regex Compilation
Bulky-Priority6824 · reddit · 2026-07-12
llama.cpp released version b9966, fixing a performance issue in the -sm tensor scenario. Previously, the decode thread would recompile 29 regular expressions for every token and tensor, leading to massive unnecessary CPU overhead.
This fix caches and reuses these regular expressions. The behavior remains unchanged, but the decode thread is much more CPU-efficient. If you are using -sm tensor in a production environment, upgrading as soon as possible is recommended.
More from Infra
- China’s AI arms race is increasingly defined by chips, data centers, and open models — BenBajarin · 2026-07-22
- Agent search bottlenecks are now about variance, not raw latency — rohanpaul_ai · 2026-07-22
- Gavin Baker argues Nvidia may be one of open source AI’s biggest supporters — GavinSBaker · 2026-07-22
- AI Power Demand Exposes US Energy Gap, Urging Shift from Scarcity to Abundance — bradneuberg · 2026-07-22
- Gavin Baker says Nvidia’s $630B figure would be system revenue, not all Nvidia’s — GavinSBaker · 2026-07-22
- A Firecracker-based platform says it can host 6,000 AI agents on one 256 GB server — maritime_sh · 2026-07-22