Changing one MCP tool description sentence lifted correct tool selection from 19/55 to 45/55
sparkignitefire · reddit · 2026-11-21
The Portravo team ran a controlled experiment on MCP tool descriptions: an account-profile tool instructed to "use this first" was hijacking requests meant for neighboring tools. With the original description, a model picked the expected tool first in 19 of 55 runs; revised descriptions reached 45 of 55; reinserting the old opening sentence dropped it back to 18 of 55. Scoring covered only the first tool selected, not end-to-end task success. Practical advice: if a tool description says "use this first," test it against ordinary requests intended for its neighbors. Full methodology published on their site.
More from coding & agent
- Founder's rebuttal to McKinsey: enterprises need a first brain, not an AI second brain — vasuman · 2026-11-21
- Devs: LLMs absorb layered complexity until context explodes; tools and MCP are too heavy — dscape · 2026-11-21
- Inkling 2 Mini on Ollama: MLX build, one-command hooks into Claude Code and OpenCode — ollama · 2026-11-21
- Unguided coding agents pass only 52% of web checks; Modern Web Guidance lifts it to 91% — rseroter · 2026-11-21
- n8n launches Agents: your existing workflows become an AI agent's toolkit — SucceededMind · 2026-11-21
- Nous Research picks winners of Hermes Agent plugin hackathon with ~1,000 entries — alexcovo_eth · 2026-11-21