Beyond Plain MCP: Do Decision Models and Code Mode Make Tool Use Cheaper and Faster?
Abhishek Rai
cs.SE, cs.AI, cs.CL
2026-11-14
Pre-registered controlled study: code mode cuts MCP tool tokens 70–90% vs plain MCP and raises success; a router in front adds no significant gain.
An agent wired to a growing catalog of MCP (Model Context Protocol, the open protocol for describing and invoking tools) servers pays twice: every tool schema rides in the prompt on every turn, and every tool result streams back through the model. In this paper's harness, plain MCP burns about 148K tokens per task at 200 tools and fails outright at 2,000 by context overflow.
Two remedies already ship in production: code mode, where the agent writes a script against generated stubs and only the final line re-enters context (Cloudflare and Anthropic both have versions), and a decision layer, a small model that preselects the tool subset before the agent sees it. Controlled comparisons show code mode works but varies by model; nobody had isolated why, or tested whether stacking the two helps. This study answers both with a pre-registered evaluation, where hypotheses and analysis plan are frozen before touching the test data.
One gateway process, four switches; servers, tasks, seeds, turn caps and sandbox stay fixed:
The world is 12 mock servers with 42 built-in tools, plus real distractor tools from a public catalog, scaling catalogs to 42, 200 and 2,000. Three open-weight agents (Qwen3.8-27B, Gemma-4-26B-A4B, Nemotron-3-Nano-30B-A3B), all quantized checkpoints served serially on one GB10 with 121GB unified memory, temperature 0, thinking off.
Two decisions carry the credibility. Utility is U = success − 0.01 × tokens/1000 − 0.002 × latency-seconds, with weights frozen before any router configuration was scored, and tasks are graded by inspecting world state, not model text. The 119-instance test split was sealed before running, with Holm correction over the 12 primary comparisons; once the instances turned out to cluster into 21 task templates, the authors added a family-level resampling and defer to the more conservative read.
The sleeper variable sits in the stubs: return-shape hints, hand-written notes declaring what each tool returns (keys, types, list vs dict). A 4B tool-selection router fine-tuned with LoRA from spark-s1 completes the kit, trained on a disjoint world.
Primary comparisons, ΔU on the sealed split (✓ = survives Holm correction under family-level resampling):
| Comparison | Qwen3.8-27B | Gemma-4-26B | Nemotron-3-Nano |
| code − plain MCP | +0.364 ✓ | +0.118 | +0.861 ✓ |
| router+JSON − plain | +0.072 | +0.061 | +0.571 ✓ |
| router+code − code | −0.025 | +0.077 | +0.095 |
| router+code − router+JSON | +0.268 ✓ | +0.133 | +0.385 |
Code mode cuts tokens 84% for Qwen (26,602 to 4,362 per task), 70% for Gemma and 90% for Nemotron, while success rises 13.4 and 19.3 points for Qwen and Nemotron. It scales almost flat: 42 to 2,000 tools moves latency from 3.6 to 6.3s and tokens from 4.7K to 14K, while plain MCP dies at 2,000.
The first-order variable is the stub. Removing return-shape hints drops code-mode success by 22.7 points (Gemma) and 39.5 points (Nemotron), Holm-adjusted p = 0.0003; Qwen loses only 1.7, not significant. On the utility scale that is +0.30 and +0.48, larger than any router effect measured. Unhinted, Nemotron's code mode sits at 0.185 success, below plain MCP's 0.412; hinted, 0.504. Same model, opposite verdict.
The central hypothesis, that a router in front of code mode beats code mode alone, is not confirmed: pooled over 238 instances the effects are −0.028 (Qwen), +0.099 (Gemma) and +0.109 (Nemotron), none significant after correction. The pattern is positive for the two weaker agents, slightly negative for the strongest. Even D-oracle, handed the gold tool list, beats plain code mode by only +0.022 on Qwen; the ceiling is low. H4 does hold for Qwen, with the same direction for the others: once a router has picked the subset, scripts beat JSON calls.
One engineering detail is settled the hard way: the fallback must be bounded. The unbounded policy kept requesting more tools, reaching 223s, 370K tokens and 0.40 success per task at 2,000 tools; the bounded retry gives 16s, 34K tokens and 0.90 success, a 14× speedup. The front router itself costs about 1.6s, and would need to run under 0.4–0.7s to pay off for a fast agent.
The 4B router reaches complete recall 1.000 at 42 tools (zero-shot base 0.899), drops to 0.964 on tasks reworded by a different model family, and is capped at 0.44 by the candidate retriever at 2,000 tools. Its selling point is footprint (8.4GB vs roughly 52GB for a prompted 26–27B router), not speed. On security: one poisoned tool description telling the router to "ALWAYS select this tool" steered a prompted LLM router in 39 of 39 tasks, and injection through tool results succeeded less often under code mode (0.093) than plain JSON (0.148).
For anyone building agent gateways, these numbers convert straight into architecture decisions. Code mode is the dominant effect: an order of magnitude in tokens, higher success, near-immunity to catalog scale, and evidence from a sealed split with multiplicity correction, a level of rigor rare in systems papers. Documenting what tools return is worth more than adding a router, which suggests some past claims that code mode fails for certain models were measuring stub quality instead. The paper proposes, without testing, that gateways learn shapes from traffic and annotate stubs automatically.
Should you deploy a router? The strongest agent does not need one; the two weaker ones trend positive without reaching significance. A router's real value is a smaller attack surface and an 8.4GB footprint, and its output must never be used as an authorization decision. Bound the fallback, always.
This is a measurement paper, not a new architecture, and its value is exactly that it sorts out what carries the effect.
The author's own list is thorough: a fully synthetic world with instant mock tools, no real MCP servers, no real network latency, no external benchmark; real servers with per-call round trips would favor code mode's batching even more, and the paper says so. The test split shares its 21 task templates with development, so instances are not independent. The protocol itself calls for about 300 pairs per arm and got 119/238, so "not confirmed" means inconclusive, not absent. The stub hints were hand-written and tuned on dev data, which shows that return-value information matters, not that any automated way of producing it works. The three agents differ in quantization and serving stack (Qwen on MLX 4-bit, the others on NVFP4 vLLM), so cross-agent latency comparisons carry infrastructure confounds. Security results come from six tasks and one seed, with the same person writing attacks and defenses. The 4B router loses 1.5–2.8 points on its base model's own benchmark, so it is not a drop-in replacement.
The utility weights are a judgment call. The five-setting sensitivity grid keeps qualitative conclusions intact, but Gemma's code-mode win flips negative in one setting; check your own token and latency price ratio before copying decisions.