FrontierML benchmark: none of 5 frontier models hits strong-result bar on 25 ML research tasks

rohanpaul_ai · x · 2026-11-20

Synthetic Sciences, a YC startup, launched FrontierML, a benchmark testing whether AI agents can do autonomous empirical ML research without ever seeing the score they're graded on. Agents get 12 hours and real H100 GPUs, must build their own progress proxies and calibrate how far to trust them, then are scored on held-out evaluations. Across 25 open-ended ML research problems, none of the 5 frontier models tested reached the strong-result bar; normalized scores: Fable 5.2 at 0.523, Opus 5.5 at 0.399, Astra 0.371, Carbon 0.353, Grok 4.7 at 0.115.

Related event: FrontierML Benchmark Launches, No Frontier Model Passes(4 posts)→

Original post →

More from coding & agent

coding & agent channel →