AutoResearchExam adds Gemini 4 Argon: frontier ML-research performance at 1/3 the cost
AlexGDimakis · x · 2026-10-21
Alex Dimakis (UCSD) shares new results from Bespoke Labs' AutoResearchExam: Gemini 4 Argon joins the Pareto curve, matching other frontier models on ML engineering performance at 2-3x lower cost.
About the benchmark:
- 29 open-ended ML research tasks measuring how fast agents improve a private test score over 24-hour auto-research runs
- Main metric is hidden-test AUARC, combining solution quality, generalization to a hidden test set, and speed — separating visible progress from validation overfitting
- Rankings shift with the evaluation time window; the analysis also compares cost efficiency and how agents allocate research effort
Takeaway: Google currently leads on cost-efficient, long-horizon open research ability.
Related event: Gemini 4 Argon Lands on AutoResearchExam at a Third of the Cost(2 posts)→
More from Models
- DeepSeek-V4.1-Flash tops Agent and WebDev Arena as #1 open model — arena · 2026-10-21
- Is 1M context plus compaction the endgame for all long-horizon AI tasks? — zmkzmkz · 2026-10-21
- Snorkel AI audit: 55% of TerminalBench 4.0 fixes had reward hacking issues themselves — ravithejads · 2026-10-21
- Ampersand v0.1 Ships via Agent API: Hosted Sessions, OpenAI-Client Compatible — gharik · 2026-10-21
- Ampersand 0.1 Nearly Ties Fable 5.1 on Collaboration Eval, Starting from GLM 5.3 — gharik · 2026-10-21
- Commentary: socially intelligent multi-user agents are the next step to real-world deployment — niloofar_mire · 2026-10-21