LLM within MCP Matters: Measuring Inefficient Resource Utilization Driven by LLMs
Minhan Cho, Soyoung Park, Kihyeon Jeong, Byeongkyu Jeon, Daejin Choi, Jinyoung Han
cs.AI, cs.CL, cs.IR
2026-08-09
54,000 trials: 9 of 24 LLMs fall below 15% hit rate with a search tool present, ignoring MCP-embedded data; without it, 23 read it at >=98%. Preference, not a capability gap.
The Model Context Protocol (MCP), introduced by Anthropic in 2024, standardizes how external data and tools are exposed to large language models. Host applications such as Claude Desktop or Cursor connect to MCP servers, which expose their capabilities through three channels: Resources (structured data fetched on demand), Tool schemas (callable functions), and Server instructions (system-prompt text describing how the tools should be used).
A common design pattern is to embed frequently needed lookup data directly into the server instructions. A legal-information server, for example, might list the 20 most-requested laws alongside their IDs (Civil Act maps to 001706). A model querying one of those laws should be able to skip search entirely and call the service with the cached ID, saving both tokens and latency. The question is whether models actually read that table, or reach for the search tool and rediscover an answer that was already in front of them.
This paper measures exactly that, and finds the problem is large enough to count as a hidden cost.
The testbed is LexLink, a production Korean legal-information MCP server exposing 26 tools. Retrieving a law normally takes two steps: eflawsearch maps a query to a law ID, then eflawservice fetches the content for that ID. LexLink's server instructions embed a table of the 20 most-requested laws and their IDs, so for any of these the efficient path is a single eflawservice(id="001706") call, skipping search.
The metric is the hit ratio phi: the fraction of trials in which the model's first tool call is eflawservice with the correct cached ID. Calling search, using the wrong ID, or anything else counts as a miss.
The authors evaluate 24 models (9 Claude, 6 Gemini, 9 GPT) across generations and tiers. Each runs under 9 conditions: 8 from a 2-cubed full factorial over three instruction-level interventions, plus a diagnostic. The three interventions:
The ninth condition, nos, removes eflawsearch entirely, leaving the embedded table as the only source of IDs. With 5 queries by 50 rounds by 24 models by 9 conditions, the study totals 54,000 trials; each cell pools 250, with a 95% binomial half-width of at most 6.2 percentage points.
The sharpest result comes from the nos diagnostic. With the search tool removed, 23 of 24 models reach a hit ratio of at least 98% (22 of them at least 99%). Nearly every model can read the table. The lone exception is gpt-4.1-nano, which only reaches 54% even without the search tool, a genuine capability deficit at the low end rather than pure preference.
Put the search tool back, and baseline hit ratio scatters from 0% to 100%, with 9 models falling below 15%. The tool's mere presence, with no change to instructions or data, is enough to override the answer written in front of the model. The authors call this failure mode a behavioral preference, distinct from a capability limitation.
Model recency does not predict efficient resource use:
| Family | Previous baseline | Current baseline |
| GPT | 4.1: 99% | 5: 1% |
| Gemini | 2.5-pro: 100% | 3-pro: 2% |
| Claude Opus | 4.5: 100% | 4.6: 63% |
The authors can only speculate on the mechanism. If post-training rewards successful tool calls on agentic benchmarks, it would teach exactly the policy observed: when a matching tool exists, call it. Provider differences in how heavily system-prompt content is weighted during that training would explain why the regression is family-specific. Separating these hypotheses needs training details providers do not publish.
In the factorial analysis, B and C have nearly identical average main effects (+19.9 and +19.8 percentage points) but very high cross-model variance (standard deviation near 24 pp), because each acts on different model profiles: B dominates for GPT-5-nano (+80.9), C for GPT-5.2 (+85.6). A single factor can also reverse: GPT-5.2 drops from 5% to 0% under B alone, yet recovers to 100% under BC, an interaction a one-factor-at-a-time design would never reveal. Stacking all three (BCD) brings 20 of 24 models to 86% or higher.
Four models score a perfect 100% across all nine conditions: cl-sonnet-4, cl-sonnet-4.5, cl-haiku-4.5, cl-opus-4.5.
The paper has a direct takeaway for three audiences.
For MCP server builders: the few-shot example (C) has a positive or negligible main effect on every model in the grid, making it the safest single change. A directive alone (B) can rescue one model and break another. For broadest coverage, combine B, C, and D. All of these are per-server patches, and brittle ones.
For host-application builders (Claude Desktop, Cursor): the real fix belongs on the host side. The paper argues the host should provide an explicit mechanism that places server instructions early in the model's deliberation before tool selection, for example by pre-injecting them as a system-priority directive or enforcing a consult-instructions-first check before each tool call.
For anyone evaluating an MCP server's capability: a server's effective capability depends not only on how it exposes data but on which client LLM consumes it. Capability reports should be paired with the client environment.
The authors list several. All evidence comes from a single server and domain (Korean law). An expanded study, to be reported separately, repeats the design on four more servers spanning geographic, academic, database, and messaging tasks, and tests a host-side mitigation; its preliminary results are consistent with the pattern here, but the paper cites them as outlook, not evidence.
The metric captures first-call efficiency rather than end-to-end task success; on this testbed the two coincide, because a correct first call retrieves the intended law directly. All trials are single-turn; a multi-turn agent may self-correct at additional token and latency cost. B, C, and D are each one concrete wording, so measured effects attach to these instantiations rather than to their categories. Finally, the conclusions cover lookup-style workflows where search is genuinely redundant; tasks where search carries real information are out of scope.
One gap remains: the authors attribute the nine low-scoring models' behavior to "behavioral preference" and concede they cannot tell a trained bias from a rational taste for verifiable tool output. The concession is honest, but it means the "why" behind the preference is never actually closed.