Claude Code MCP benchmarked GPT-5.6, DeepSeek, GLM and local Qwen across 198 hidden tests
MeetStraight1899 · reddit · 2026-07-21
I built an MCP server so Claude Code could delegate tasks to GPT-5.6, DeepSeek, GLM, and a local Qwen, then benchmarked all of them against Claude itself with 198 runs and hidden test suites.
- The setup exposes two tools, listmodels and delegatetask, and works with any MCP-capable agent.
- Models were routed through Codex CLI, DeepSeek v4, z.ai’s coding plan, and a local Qwen on LM Studio over LAN.
- The author ran 6 task stations × 11 models × 3 rounds.
- Single-run results were misleading: some “wins” were flukes, while a few weaknesses repeated across rounds.
- The GPT-5.6 Codex family reportedly went 54/54 across all technical runs.
The core takeaway: as an orchestrator, Claude can safely hand off more work than expected, but only if you measure with repeated hidden tests instead of trusting one-off success.
Related event: Multi-Model Coding Agent Test: Cheaper Models Fail Due to Stale Knowledge(2 posts)→
More from coding & agent
- Alchemy adds effect/Workflow support across Durable Objects, Workflows and Lambda Durable Functions — samgoodwin89 · 2026-10-27
- Alchemy's Lambda Web Functions run plain Node web servers on Lambda, no OpenNext needed — samgoodwin89 · 2026-10-27
- Alchemy Domain brings domain-driven design and story testing to Effect backends — samgoodwin89 · 2026-10-27
- Alchemy makes all ~2,850 AWS bindings work from any cloud, not just S3/Lambda/DynamoDB — samgoodwin89 · 2026-10-27
- Production AI agent scored 95% robustness at baseline, 37% under adversarial attack — njyx · 2026-10-27
- Chaining Grok Bot and Codex to detect and mass-block AI reply accounts — vista8 · 2026-10-27