Stanford Study Finds LLM Judges Favor Their Own Medical Answers
A Stanford study of 13,760 judgments across 8 models from OpenAI, Anthropic, Google and Qwen found that LLM judges systematically prefer their own answers when evaluating responses to 620 real medical questions, raising concerns about the reliability of LLM-based evaluation.
2026-11-18 ~ 2026-11-18 · 2 related posts
- Stanford finds all 8 LLM judges favor their own medical answers, skewing benchmarks — rohanpaul_ai · 2026-11-18
1 near-duplicate retellings: rohanpaul_ai