dottxt tool-call index shows LFM3-1.7B broken tool calls jump from 7.19% to 15.96% across providers

remilouf · x · 2026-11-18

.TXT (dottxt) published its Tool-call index at providers.dottxt.ai, tracking broken tool-call rates across models and inference providers. Latest finding: Liquid AI's LFM3-1.7B breaks 7.19% of tool calls on Liquid's official fp8 endpoint but 15.96% on Liquid fp8's free tier — the same model, twice the failures. The index covers dozens of models (DeepSeek, Kimi, GLM, Qwen, MiniMax, Mistral, Nemotron, etc.) across providers. Key takeaway: tool-calling reliability depends on the provider and deployment, not just the model — agent builders should factor the serving layer into their selection.

Original post →

More from coding & agent

coding & agent channel →