Artificial Analysis adds reasoning-effort filters: effort level swings scores and cost 8x
ArtificialAnlys · x · 2026-11-19
Artificial Analysis has rolled out reasoning-effort comparison across its site: effort filters on every chart, homepage toggles between max effort and API defaults, and a Release Comparison tool supporting up to five models across 10 effort levels on intelligence, pricing, and time per task.
Sample data comparing Claude Fable 5.2 and GPT-6.2 Astra shows effort level matters enormously: Fable 5.2's Intelligence Index climbs from 46 (low) to 65 (max) while cost per task jumps from $3.10 to $26.54; HLE-Diamond goes 60.8% → 80.2%. Key takeaway: a premium model at low effort can underperform a cheaper model — choosing the effort level may matter more than choosing the model.
More from Models
- Independent sandbox evaluation validates Xlue team's model performance — XiongChenyan · 2026-11-19
- Ultrafast mode review: great for math proofs, not recommended for coding — gnukeith · 2026-11-19
- Gemini 4 "Neon" spotted in Google AI Studio code alongside mysterious "Cobalt" model — testingcatalog · 2026-11-19
- Oxford paper: post-training makes LLM worldviews bleaker, models converge on 'spiritual bliss' — twiecki · 2026-11-19
- Artificial Analysis adds site-wide reasoning-effort filters for model benchmarks — ArtificialAnlys · 2026-11-19
- GPT 6.2 takes SOTA on ClutterBench, a leaderboard long dominated by Claude models — sandersted · 2026-11-19