Study Reveals Heavy Contamination in Major LLM Benchmarks

A systematic audit of popular LLM benchmarks like GPQA and MMLU-Pro reveals severe data contamination, with up to 12% of questions flawed. Scores skyrocket to near-perfect after fixing these issues, challenging the validity of current evaluations.

2026-07-29 ~ 2026-07-29 · 2 related posts

• Audit finds up to 12% of GPQA, MMLU-Pro, and MMMU-Pro questions were broken — pawofdoom · 2026-07-29
• Study Reveals 12% of Questions in Major LLM Benchmarks Are Broken, Skewing Scores — pawofdoom · 2026-07-29