Claude Code MCP benchmarked GPT-5.6, DeepSeek, GLM and local Qwen across 198 hidden tests
MeetStraight1899 · reddit · 2026-07-21
I built an MCP server so Claude Code could delegate tasks to GPT-5.6, DeepSeek, GLM, and a local Qwen, then benchmarked all of them against Claude itself with 198 runs and hidden test suites.
- The setup exposes two tools, listmodels and delegatetask, and works with any MCP-capable agent.
- Models were routed through Codex CLI, DeepSeek v4, z.ai’s coding plan, and a local Qwen on LM Studio over LAN.
- The author ran 6 task stations × 11 models × 3 rounds.
- Single-run results were misleading: some “wins” were flukes, while a few weaknesses repeated across rounds.
- The GPT-5.6 Codex family reportedly went 54/54 across all technical runs.
The core takeaway: as an orchestrator, Claude can safely hand off more work than expected, but only if you measure with repeated hidden tests instead of trusting one-off success.
Related event: Multi-Model Coding Agent Test: Cheaper Models Fail Due to Stale Knowledge(2 posts)→
More from coding & agent
- Security tip: use AI agents to build a NIST CSF knowledge base for incident response — prasanna_says · 2026-10-26
- Google Cloud Adds Self-Learning Context Enrichment to Knowledge Catalog for AI Agents — rseroter · 2026-10-26
- Cloudflare Previews Spend Limits for Workers and Durable Objects, Early Access Open — ritakozlov · 2026-10-26
- Shopify design lead on AI-era workflow: designers are becoming full-stack editors — KlausCodes · 2026-10-26
- Nace AI launches Drex for Search, cutting agent steps 62% and tokens 56% — testingcatalog · 2026-10-26
- Ultrametric open-sources Open Startup Repo to let AI agents run startup operations — garrytan · 2026-10-26