When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
Wenda Xu, Sweta Agrawal, Vilém Zouhar, Markus Freitag, Daniel Deutsch
cs.CL, cs.AI
2025-10-01
Gemini 2.5 Pro, GPT-4.1 and Claude Opus 4 each rank first on translation tests they generate and score; testset and evaluator bias compound in low-resource languages.
Public leaderboards fill up in months. Human-written test sets are slow and expensive, especially for languages with little labeled data. A popular workaround is LLM-as-a-benchmark: one model writes the test inputs (LLM-as-a-testset) and scores the outputs (LLM-as-an-evaluator).
Judge self-preference is already documented. This paper isolates the missing half: the test items themselves can be written in the generator's house style. Google and ETH Zurich use machine translation as the main testbed, because MetricX-QE, WMT25 human rankings, and FLORES-200 human sources let them split generation bias from evaluator bias.
The three systems are Gemini 2.5 Pro, GPT-4.1, and Claude Opus 4, still separable by WMT25 human rankings. Language pairs are low-to-medium resource: Bemba, Aymara, Luo, and Kurdish, both into and out of English. Two hundred items per direction, temperature 0. Open-source multilingual models failed to write usable low-resource tests and were dropped.
Self-bias is the model's average rank of itself minus the average rank assigned by the other two. More negative means stronger self-promotion. Ranks, not raw scores, keep metrics comparable. Peer consensus is a proxy for quality, so the quantity is relative self-preference, not error versus an absolute ground truth.
Three settings:
A source-only variant later removes co-generated references, so translatability bias (picking sentences the model can translate) is not mixed with generation style. Diversity is measured with chrF@K within versus across models. Degeneration is a text with at least ten repeating 4-grams.
Under the full benchmark, every model ranks itself first. Mean ranks over four XX→English directions (lower is better): Gemini-as-benchmark puts Gemini at 1.222, GPT at 1.525, Claude at 1.564. GPT-as-benchmark puts GPT at 1.222. Claude-as-benchmark puts Claude at 1.123. On Bemba→English, Claude's self-bias hits -0.617 even though the other two disagree.
The two sources compound. Mean XX→English self-bias:
| Setting | Gemini | GPT | Claude |
| Testset only | -0.124 | -0.239 | -0.093 |
| Full benchmark | -0.261 | -0.294 | -0.550 |
Source-only generation still shows a negative diagonal; Claude reaches -0.763 on Bemba→English. Co-generating a reference improves MetricX (Gemini on Bemba→English: 8.49 down to 2.73), so models do pick easier sentences. Style bias remains after that confound is removed.
Within-model chrF@K exceeds cross-model scores: on Aymara, Gemini is 37.78 versus about 32 across models. Gemini degenerates on 22.3% of sources, GPT-4.1 on 5.8%, Claude on 0%. The 50 most diverse sources (Min chrF) cut testset-only self-bias versus the 50 least diverse (Max chrF): Gemini -0.170 vs -0.240, Claude -0.205 vs +0.045.
Direction is the sharper split. Gemini's self-bias is -0.15 on English→Bemba and -0.59 on Bemba→English. English TTR profiles nearly overlap (Cohen's D under 0.2); low-resource profiles diverge (D above 0.5). Chatbot Arena repeats the pattern: English and Korean near zero, Bemba and Aymara much larger. Ranking Gemma3-27B, Mistral-Large-2411, and Qwen3-32B with these three frontier models keeps most bias inside 0.05. A large quality gap still yields a stable order.
Anyone using a model to build its own leaderboard needs a use/don't-use map. High-resource languages and a large capability gap: still usable. Low-resource languages and near-peer models: ranks can flip. Diversity filtering trims testset bias; under the full pipeline, GPT's evaluator bias can swamp the filter. Automated benchmarks are not a neutral field.
Only three closed frontier models and two tasks. The "true" ranking is peer consensus, not large-scale human labels. MetricX-QE, outside all three families, still shows self-bias, so the finding is not an artifact of peers grading peers. Shared taste among judges could still be counted as quality. Diversity filtering is significant for all three under testset-only scoring; under the full benchmark it is not significant for GPT. High-resource pairs sit near ceiling, so self-bias is hard to measure there. That does not mean evaluator bias is gone.