dottxt launches Tool-call index: same open model fails tool calls up to 13x more on some providers
remilouf · x · 2026-11-11
dottxt launched the Tool-call index (providers.dottxt.ai), measuring how often the same open-weight model breaks tool calls across inference providers — with gaps up to 13x or more between the best and worst provider. Author remilouf argues that when your agent runs on an open-weight model, reliability hinges on a provider choice you may not have made deliberately, and "this model is bad at tool calling" may actually be a provider problem. The weekly-updated index covers DeepSeek, Gemma, Llama, MiniMax, Mistral, Kimi, Nemotron, gpt-oss, Qwen and more: e.g. gpt-oss-120b ranges from 0.04% to 58.42% error rate, Llama 3.1 8B from 1.78% to 51.30%, Kimi K3 from 0.00% to 0.72%. It exposes an llms.txt for agents and a one-line skill install (npx skills add https://providers.dottxt.ai --skill choose-provider) so agents can pick the best provider themselves.
More from coding & agent
- LlamaIndex to demo parsing hundreds of pages of insurance medical records with LlamaParse — llama_index · 2026-11-11
- LlamaParse Live Demo: Parsing Hundreds of Pages of Faxed Insurance Medical Records — llama_index · 2026-11-11
- Cloudflare ships experimental Agents SDK example running the Codex agent loop on Workers — dinasaur_404 · 2026-11-11
- Nous Hermes agent turns an idea into printable CAD, iterating on slide-fit lid mechanism — Teknium · 2026-11-11
- EMNLP survey organizes LLM routing into predictive, adaptive and generative paradigms — kalyan_kpl · 2026-11-11
- Telegram Now Turns Any Message Into an App Under 4KB With Built-in Payments — CurieuxExplorer · 2026-11-11