Benchmark cheating debate: scaling a 3B open model to ~60B explains the scores
RexDouglass · x · 2026-11-17
In a debate over suspected benchmark gaming, jonaskreutz01 argues the model's performance matches scaling a 3B open-source model to 60B with better data, so no cheating is likely. RexDouglass says commercial endpoints are 'whatever works' but private benchmarks should be linked, distinguishing detecting AI text going forward from identifying pre-2020 text (much easier with indexed historical corpora). bengolub notes it's far from memorizing a magic 'pre-2020' tell.
Related event: Benchmark Cheating Debate: Performance Explainable Without Foul Play(2 posts)→
More from Models
- Teaser: gemini-4-neon is no lightweight flash model, hints insider — lyraxana · 2026-11-17
- OpenAI urged to release rumored model allegedly cracking Navier–Stokes millennium problem — haider1 · 2026-11-17
- Verus Proof Index: new formal-verification benchmark evaluates 25 models' agentic proofs — fhuszar · 2026-11-17
- Asking ChatGPT and Claude to play NeurIPS reviewers — they reject the papers — burny_tech · 2026-11-17
- Physician slams OpenEvidence's "lowering all-cause mortality" claim; Eric Topol amplifies — EricTopol · 2026-11-17
- wojak-bench: Community Arena Ranks AI Models by Whose 3D-Printed Figure Matches the Drawing Best — osanseviero · 2026-11-17