Tool calls slow LLM APIs up to 32x per token, worst provider hits 41x

remilouf · x · 2026-11-18

The dottxt team shared benchmarks on the latency cost of tool calling: generating tool calls can be dramatically slower per token than plain string output — up to 32x slower per token in their tests, and the slowdown affects the entire batch, not just one request.

An accompanying chart shows per-token overhead across providers: the worst reaches 41x, with many in the 12–17x range, and even the best providers showing 10%–77% extra latency. The team argues tool calling quality directly impacts users, but providers are paying a hefty, often underappreciated performance price for it.

Original post →

More from Infra

Infra channel →