FrontierML benchmark tests whether agents can spend money to scale ML performance
akbirthko · x · 2026-10-14
Induction Labs introduced FrontierML, a benchmark testing how well agents use money to scale model performance across 21 challenging, end-to-end ML tasks — the judgment human researchers apply when deciding compute/data spend and which experiments to run.
Each agent gets a wallet ($10/$50/$200) plus API costs. Key results:
- Human researchers still lead, scoring 66 at the $200 budget
- Claude Opus 5.5 is the strongest model at 60 with $200
- Claude Sonnet 5.5, GPT-6 Astra, and Claude Fable 5.1 form the second tier (40s)
- Most smaller models (Gemini 3.8 Flash, Grok 4.7, Muse Spark 1.3) score under 10 with little gain from larger budgets
The benchmark targets the researcher-judgment bottleneck on the path to automated ML research.
More from coding & agent
- Intent ships PR review with inline diffs, Linear integration, board views and more — LukeW · 2026-10-14
- Gemini 4 Argon ships in GitHub Copilot, beating prior Gemini models on every coding benchmark — rseroter · 2026-10-14
- Microsoft Copilot's real shift: from asking AI to delegating work to it — WirelessLife · 2026-10-14
- "Software is dead" debunked: vibe-coding on the fly costs tokens, compute and 15-minute waits — bendee983 · 2026-10-14
- gittifacts: an agent-native git platform built in a week on Cloudflare's challenge — threepointone · 2026-10-14
- Crowd of agents push integer multiplication bounds with 'slop-search', kappa records falling every few minutes — iskander · 2026-10-14