GPT-6.1 Sol Benchmarked Across All Effort Levels; Atra Faster but Pricier
PawelHuryn · x · 2026-10-01
Benchmarker PawelHuryn published results for GPT-6.1 Sol across all reasoning effort levels on his benchmark site, with n=3 runs for the max and xhigh levels and n=2 for the rest; additional n=3 runs are still in progress.
Compared with Astra: Astra is notably more expensive, but faster and spends fewer turns on the most challenging tasks. Live scores, caveats, and more models are available on the site.
More from Models
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- Nathan Lambert calls out Ethan Mollick for 'selectively ignoring' Kimi in model comparisons — natolambert · 2026-11-16
- Stealth Anthropic model spotted in Devin build as METR chart shows frontier pace up 1.4x — daniel_mac8 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- Unsloth Q8 crushes default Q3 in a self-hosted test over 331 financial PDFs — No_Cartographer3953 · 2026-11-16