Independent check verifies Horizon's SOTA claims on held-out synthetic benchmarks

maksym_andr · x · 2026-11-19

Skeptical that Opus 5.5 overclaimed its contributions, the team manually verified performance on newly generated synthetic datasets unseen by the models — and found them genuinely good. Their report found no evidence of overfitting or benchmark gaming, noted Horizon picks checkpoints by validation F1 (a better proxy even when reported test scores look worse), and using MiMo V2.6 Pro-generated held-out benchmarks across 4 model families found the models often SOTA for their size on narrow tasks, e.g. Injection Guard variants on the Pareto front.

Original post →

More from Models

Models channel →