Haiku falls below 90% past 10 MCP tools; five server patterns from 15 systems

MCP Server Architecture Patterns for LLM-Integrated Applications

Carson Rodrigues, Oysturn Vas

cs.SE, cs.AI

2026-06-29

Five MCP server patterns from 15 systems. Haiku 4.5 picks the right tool 91% of the time at 10 tools and 87% at 15; Sonnet 4 stays above 90% until 20-30 tools.

What problem this solves

MCP pulls tools, resources, and prompts out of the model and into a JSON-RPC protocol. One server can talk to Claude, GPT, or Gemini. Hundreds of community servers showed up within months of the November 2024 spec. Architectural guidance for production did not: how to split tools, when server-side session state is justified, how to aggregate a fleet. Hou, Hasan, and Guo have already covered MCP security and ecosystem scale. Nobody had catalogued the recurring server-side structures under the constraint that the client is an LLM reading natural-language descriptions.

Celabe's ANSYR voice platform has run five production MCP servers since late 2024. Add ten public servers from the official modelcontextprotocol/servers registry and the derivation corpus is fifteen. This ICSME industry-experience paper extracts five patterns and four anti-patterns from that set.

Method

Each pattern is written in the Gang of Four form: context, problem, solution, consequences, known uses. Every pattern has a classical ancestor (Repository, Facade, Session, Proxy, Adapter). The delta is the LLM client: the model picks operations by reading descriptions, not by browsing docs.

Five patterns:

Four anti-patterns: the God Tool (doanything(action, params)), unsanitized resource content, synchronous long-running jobs, and missing or tautological tool descriptions.

Reliability is measured separately. Fifty-four held-out servers, described in architecture-neutral language. Two independent LLM raters (Haiku 4.5 and Sonnet 4, temperature 0) label each one.

Results

SetupNumber
Inter-rater Cohen's κ0.76 (95% CI 0.62-0.88), 81.5% raw agreement
Agreement with author labelsHaiku 68.5%, Sonnet 75.9%
Haiku 4.5 tool pick91% at 10 tools, 87% at 15, median 245 ms
Sonnet 4 tool pick95% at 10 tools (410 ms), still ≥90% at 20, below at 30
stdio / loopback streamable-httpp50 0.01 ms / 0.39 ms (measured, N=100)
same-region remote / aggregator remotep50 30.4 ms / 62.4 ms (modeled, not measured)

The tool-count numbers are observational, from ANSYR production logs in Q1 2025. Two hundred session turns per bucket, human QA as ground truth, Wilson 95% intervals about ±4 points. This is a retrospective analysis, not a new controlled experiment.

Disagreements cluster at three boundaries. Statefulness is invisible from a capability list, so every stateful server is tagged Tool Orchestrator. Domain validation is equally invisible, so adapters split between Orchestrator and Gateway. Retrieval-style orchestrators get relabeled Resource Gateway. The authors now treat statefulness and domain logic as cross-cutting attributes, not mutually exclusive categories.

Transport is simpler. Protocol overhead is drowned by network RTT. Once the path crosses a host, stdio versus HTTP barely matters. The real choices are co-location and whether a Proxy Aggregator adds another hop.

Why it matters

A working vocabulary for people writing MCP servers. Read-mostly data goes through a Gateway with sanitization. Cross-system actions become Orchestrators. Reach for a Session server only when a turn depends on earlier state, and budget for reaping. Aggregate a fleet with the scoped variant, not a static merge. Keep a single context around 10-15 tools. Tool descriptions are load-bearing. Treat them like code.

The same patterns are maintenance seams. An Adapter absorbs upstream API churn. An Aggregator is the single place to auth, version, and audit. A Gateway confines schema migrations to one layer.

Limitations

The derivation corpus is fifteen servers. Guo et al. counted more than 8,000 public ones; a stratified replication was not attempted. Classifiers saw function descriptions, not running servers. Both raters are Claude models and may share blind spots; human dual-coding is future work. Three of five latency rows are modeled. The tool-count study is one voice platform's telemetry; raw session logs are not released, only a csv. The first author is employed by Celabe.

Terms

Source

What people are saying

Related papers

All paper explainers