Marin admits substantial benchmark contamination slipped past its decontamination pipeline
dlwh · x · 2026-11-18
The Marin team disclosed that they found substantial contamination from common pretraining benchmarks in Mochi 535B-A23B that their decontamination procedure failed to catch.
- This limits which model comparisons they can fairly report
- They documented what happened and linked a full investigation in the blog post
Following the release of 57 intermediate checkpoints, this is another transparency move — and a notable real-world case that decontamination pipelines can silently fail.
More from Research
- Lean community certifies Riemann proof; honeypot fake keys expose breaches at three AI providers — LeviTurk · 2026-11-22
- Riemann hypothesis 'proven by Turkish mathematician using AI' claim mocked as the 'Organic Turk' — CatAstro_Piyush · 2026-11-22
- Andriy Burkov releases The Hundred-Page Language Models Book, hands-on with PyTorch — burkov · 2026-11-22
- Generative AI meets cardiology: Dagstuhl seminar bridges ML and clinical practice — StephanMandt · 2026-11-22
- SubRoformer hits SOTA music source separation with DSP subband filters — affige_yang · 2026-11-22
- Emily Riehl releases Lean game teaching dependent type theory at intro-to-proofs level — abhi9u · 2026-11-22