MCP testing: one question went from 437k to 78k input tokens — cost is driven by round trips
loookashow · reddit · 2026-09-08
A detailed engineering postmortem on agent-testing an MCP server: conformance testing found nothing, while 18 real-agent scenarios found five issues in a week. Key lessons: hosted connectors resend the whole conversation every tool-loop iteration, so cost equals returned payload times remaining turns (one scenario dropped from 437k to 78k input tokens); Anthropic's connector caps at 10 tool iterations and returns pauseturn with HTTP 200, which naive clients log as success; JSON-RPC notifications are defined by a missing id, not the method name; and checks should allow agent exploration — a "no tool errors" assertion wrongly failed scenarios where guessing and recovering was the system working. Page size now defaults to 5, capped at 100, published in the tool schema.
More from coding & agent
- LangChain and Tavily to co-host NYC event on production agents using live web data, Nov 12 — LangChain · 2026-10-26
- SentinelForge: evidence-driven AI software repair with human-in-the-loop verification — Byte_Forge_2811 · 2026-10-26
- Dev hacks reMarkable e-ink tablet to let Opus draw and animate — Stefania_druga · 2026-10-26
- Godot games are trivially decompilable — compile C# to native DLL for real protection — mflux · 2026-10-26
- Techhalla shares full 2D-to-3D workflow with Opus 5.5 and Kling 4.0 prompts — techhalla · 2026-10-26
- Jason Furman: Building Econvitals pre-AI would need ~50 people, 3 years, $20M — HarrySurden · 2026-10-26