Changing one MCP tool description sentence lifted correct tool selection from 19/55 to 45/55

sparkignitefire · reddit · 2026-11-21

The Portravo team ran a controlled experiment on MCP tool descriptions: an account-profile tool instructed to "use this first" was hijacking requests meant for neighboring tools. With the original description, a model picked the expected tool first in 19 of 55 runs; revised descriptions reached 45 of 55; reinserting the old opening sentence dropped it back to 18 of 55. Scoring covered only the first tool selected, not end-to-end task success. Practical advice: if a tool description says "use this first," test it against ordinary requests intended for its neighbors. Full methodology published on their site.

Original post →

More from coding & agent

coding & agent channel →