MMTEB Leaderboard Faces Training Data Contamination Controversy
The MMTEB leaderboard is under scrutiny for potential training data contamination. Observers noted that some top-performing models, like the Harrier series, might be compromised on official tasks like SpartQA and MIRACL, leading to misleading evaluation results.
2026-07-15 ~ 2026-07-15 · 3 related posts
- MMTEB Leaderboard Suspected of Training Set Contamination — antoine_chaffin · 2026-07-15
- MMTEB Leaderboard Suffers from Training Set Contamination — antoine_chaffin · 2026-07-15
- Harrier Sparks Controversy on MMTEB Leaderboard — antoine_chaffin · 2026-07-15