Benchmarking MCP Server Context Overhead
Crazyblind · reddit · 2026-07-11
The author open-sourced a CLI, lap, to measure the context window tax on agent-facing interfaces, covering MCP servers, OpenAPI menus, and local tool stacks.
Core Findings
- Among 20 public MCP servers, the Notion official server consumes 21,411 tokens just for tool definitions in a single session.
- Firecrawl takes about 18,511 tokens.
- Connecting all 20 servers eats up roughly 64k tokens before startup.
- Despite having only one tool, sequential-thinking requires 921 tokens due to its lengthy description.
Further Experiments
- Compressing the same tools into tighter signatures drops the total cost to 3,100 tokens, a roughly -95% reduction.
- lap stack reads your Claude Desktop / Cursor configurations to calculate startup costs.
- lap lint --mcp "<command>" detects missing tool descriptions, undocumented parameters, overly long definitions, and missing required fields.
Results & Impact
- The author launched a monthly-updated public API leaderboard showing that directly converting OpenAPI to MCP menus accounts for 11.2M tokens in scale, meaning 82% of the cost appears recoverable.
- They also tested the "layered schema" proposal from the spec, finding that the discovery tier saves an average of 87%.
- The project is MIT licensed with no commercial wrapping; the author notes it typically spots issues within ten seconds, and upstream maintainers usually fix them quickly.
Related event: MCP Tool Schema Overload Impairs Agent Accuracy(3 posts)→
More from coding & agent
- Tenable and AWS launch a Black Hat build event for open-source security agents and MCP servers — Dave_Maynor · 2026-07-22
- Codex helps build Valdiluce, an open-world game with climbing, gliding and gondolas — Dimillian · 2026-07-22
- HeyGen adds a media-sourcing skill for coding agents with 75k images and 10k tracks — HeyGen · 2026-07-22
- Agent search bottlenecks are now about variance, not raw latency — rohanpaul_ai · 2026-07-22
- LangSmith adds tracing for Pipecat, LiveKit, OpenAI Realtime, and Gemini Live — LangChain · 2026-07-22
- An MCP server signs every AI agent tool call into a verifiable Merkle chain — Funky_Chicken_22 · 2026-07-22