FrontierML benchmark: none of 5 frontier models hits strong-result bar on 25 ML research tasks
rohanpaul_ai · x · 2026-11-20
Synthetic Sciences, a YC startup, launched FrontierML, a benchmark testing whether AI agents can do autonomous empirical ML research without ever seeing the score they're graded on. Agents get 12 hours and real H100 GPUs, must build their own progress proxies and calibrate how far to trust them, then are scored on held-out evaluations. Across 25 open-ended ML research problems, none of the 5 frontier models tested reached the strong-result bar; normalized scores: Fable 5.2 at 0.523, Opus 5.5 at 0.399, Astra 0.371, Carbon 0.353, Grok 4.7 at 0.115.
Related event: FrontierML Benchmark Launches, No Frontier Model Passes(4 posts)→
More from coding & agent
- Cognee builds company ontologies for agents with open-weight models and Postgres, no graph DB — philipvollet · 2026-11-20
- Developer uses Antigravity agent swarm to auto-build two full pinball games — AI_Andrew · 2026-11-20
- Ridges launches Ridgeline: label a GitHub issue, an agent fleet ships the PR — markjeffrey · 2026-11-20
- Claude drew an entire flyable game world with pure code — no images, no 3D models — prasenx · 2026-11-20
- Fixing mobile input auto-zoom: a Shadcn refinement workflow with Impeccable skills — bennash · 2026-11-20
- DeepSeek V4.1 Flash pushed from broken to 284+ t/s on RTX PRO 6000, +78.7% over vLLM — SergioPaniego · 2026-11-20