Same model, 15x more broken tool calls: dottxt's provider tool-call index

remilouf · x · 2026-11-17

dottxt's Tool-call Index shows the same model can have wildly different tool-call breakage rates across inference providers. MiniMax M3.1 Flash: 0.10% broken calls on Parasail (fp8), but up to 10.28% on Wafer's nvfp4 — a 15x gap driven by the provider, not the model.

The index is measured over 3.4M tool calls per day across 12 MiniMax providers, also listing pricing ($0.15/$0.60 per MTokens), throughput, and uptime, updated weekly. Takeaway: when picking a model for agents, the provider's quantization and serving quality can matter more than the model itself.

Original post →

More from coding & agent

coding & agent channel →