Creator spends $25,000 benchmarking reasoning_effort across models, open-sources trajectories
zainhas · x · 2026-10-29
zainhas' team spent $25,000 testing what reasoningeffort setting to use across models, and open-sourced all the test trajectories so others can dig in. Morgan Linton called it a great breakdown of model/effort tradeoffs.
More from Models
- Artificial Analysis Intelligence Index v5: 60% now uses private test sets — ajratner · 2026-10-29
- Suhail launches Particle: particle-1 matches performance with ~45% fewer tokens — chrisalbon · 2026-10-29
- Musk praises Demis's Gemini 4 Carbon, SOTA on key benchmarks — elonmusk · 2026-10-29
- Exploring a whole repo with a 1M-context open model cost just $0.63 — HowDevelop · 2026-10-29
- ARC-AGI scores follow power law v2 = v1^4.53, explaining 98% of variance — typewriters · 2026-10-29
- Commentary: Google is now shipping frontier models every two weeks — heyshrutimishra · 2026-10-29