Three LLMs review the same diff via MCP: Claude 83, GPT-5.6 32, Gemini 80
lumir2026 · reddit · 2026-09-04
A developer open-sourced a multi-LLM code review panel (MIT) where Claude Code hosts Gemini and GPT as MCP servers to cross-review the same diff: Claude 83, GPT-5.6 32, Gemini 80 — two models said the code was fine, one said it was broken.
Key findings:
- Same-vendor review isn't independent: Claude's 299-line spec passed a Claude-only multi-perspective review, but a cross-vendor pass found 12 issues, all accepted
- Every real bug was caught by a live MCP call; none by unit tests
- A crashed reviewer used to score 0 and drag the weighted average to 57.75/FAIL with no degradation flag
- Gemini's first live call reported itself as 'Claude' — models are unreliable narrators, so identity checks now trust the tool called, not self-reported names
- A 9-value enum without allowed values in the prompt caused both vendors to invent values; the Gemini leg never even receives the base prompt due to relay behavior
Best lesson: if you relay prompts through an MCP server, assert on the string you actually hand the tool, not on the file it lives in — a substring check proves existence, not delivery.
More from coding & agent
- Clanker Cloud Opens Free Web Trial With $20 Worth of Starter Credits — tekbog · 2026-09-04
- Clanker Cloud Lets Anyone Build and Host Agents, With an Enterprise Sales Cautionary Tale — tekbog · 2026-09-04
- Running Grok bots like a company: AI project manager coordinates specialist agents — FinanceYF5 · 2026-09-04
- AI Finds Bugs Faster Than It Fixes Them: Engineers Grapple With CVE Backlogs — _jaydeepkarale · 2026-09-04
- Should Agents Govern Themselves? AAV Adds an External Action-Verifier Layer — CarlosMarreroAAV · 2026-09-04
- Dev spends $40 on classifier evals to cut costs: 'hard to use AI when you can't afford intelligence' — zeeg · 2026-09-04