Claude Code MCP benchmarked GPT-5.6, DeepSeek, GLM and local Qwen across 198 hidden tests
MeetStraight1899 · reddit · 2026-07-21
I built an MCP server so Claude Code could delegate tasks to GPT-5.6, DeepSeek, GLM, and a local Qwen, then benchmarked all of them against Claude itself with 198 runs and hidden test suites.
- The setup exposes two tools, listmodels and delegatetask, and works with any MCP-capable agent.
- Models were routed through Codex CLI, DeepSeek v4, z.ai’s coding plan, and a local Qwen on LM Studio over LAN.
- The author ran 6 task stations × 11 models × 3 rounds.
- Single-run results were misleading: some “wins” were flukes, while a few weaknesses repeated across rounds.
- The GPT-5.6 Codex family reportedly went 54/54 across all technical runs.
The core takeaway: as an orchestrator, Claude can safely hand off more work than expected, but only if you measure with repeated hidden tests instead of trusting one-off success.
Related event: Multi-Model Coding Agent Test: Cheaper Models Fail Due to Stale Knowledge(2 posts)→
More from coding & agent
- Dev builds talk on guardrails workflow for shipping AI-written code without reading it — TejasKumar_ · 2026-09-11
- banteg: Codex auto-review has regressed, blocking steps needed to complete authorized tasks — banteg · 2026-09-11
- A doc-anchored agent workflow: you write, the agent only critiques and finds disagreements — lucasmeijer · 2026-09-11
- SymKit MCP: 44 tools for AI agents to verify symbolic derivations — Foreign-Specific-604 · 2026-09-11
- GitHub Copilot team routes user bug reports to an AI agent via Slack — marlene_zw · 2026-09-11
- Scanning 23 agent sessions, a dev found 3 silent failure modes in memory systems — No_Advertising2536 · 2026-09-11