Cohere benchmarked North Small Translate on WMT26, released after training, to avoid contamination
cohere · x · 2026-10-04
In part 5/6 of its North Small Translate thread, Cohere explains its benchmarking approach: the team used WMT26 benchmarks, which were released after the model was created, ensuring the model could not have been trained on the test set.
This is a direct answer to common benchmark-contamination concerns around open translation models.
Related event: Cohere Open-Sources North Small Translate Model(3 posts)→
More from Models
- Three Google flagship models in eight days fuel speculation about hidden RSI work — haider1 · 2026-11-19
- Mystery Model 'Gestalt' Tops New Frontier Design Benchmark by 214 Points — soleio · 2026-11-19
- mrfakename releases abliterated GLM-5.3-Flash and Qwen3.8-Flash-Next — mrfakename0 · 2026-11-19
- Physician: AI competition now measured in weeks, GPT-6 versions 7 days apart — ramos_casals · 2026-11-19
- micro1's flow qc model screens training data for benchmark leaks before release — Exp_Mark · 2026-11-19
- Leaker predicts new DeepSeek paper this week, scheduled for November 11 — teortaxesTex · 2026-11-19