Benchmark scores mean little without training-data transparency, researcher warns

Michael_D_Moor · x · 2026-10-01

AI researcher Michael Moor argues that test-set performance on any benchmark carries little meaning unless it's clear what the model was trained on — models may be partially trained on the very test sets used to evaluate them. Benchmaxxing, he says, makes all non-hidden benchmarks hard to trust, and hype over new model scores is usually unwarranted.

Related event: Community Questions Reliability of AI Benchmark Scores(3 posts)→

Original post →

More from Research

Research channel →