Benchmark cheating debate: scaling a 3B open model to ~60B explains the scores

RexDouglass · x · 2026-11-17

In a debate over suspected benchmark gaming, jonaskreutz01 argues the model's performance matches scaling a 3B open-source model to 60B with better data, so no cheating is likely. RexDouglass says commercial endpoints are 'whatever works' but private benchmarks should be linked, distinguishing detecting AI text going forward from identifying pre-2020 text (much easier with indexed historical corpora). bengolub notes it's far from memorizing a magic 'pre-2020' tell.

Related event: Benchmark Cheating Debate: Performance Explainable Without Foul Play(2 posts)→

Original post →

More from Models

Models channel →