Gemini 3.7 Flash tops new AA-AnalystAgent quantitative analysis benchmark
_philschmid · x · 2026-08-19
Gemini 3.7 Flash achieved the #1 spot on the new AA-AnalystAgent benchmark by Artificial Analysis. The benchmark tests AI agents on 80 real-world quantitative analysis tasks across 14 domains like finance and healthcare, simulating the daily work of business and data analysts. Models run inside an isolated Python 3.12 sandbox using spreadsheets and documents. The ranking metric is pass@5, requiring correct answers across all 5 independent runs to emphasize reliability over one-shot success.
Related event: Gemini 3.7 Flash Tops AA-AnalystAgent Benchmark(5 posts)→
More from Models
- Muse Spark vs Qwen and GLM: US open models still trail on agentic tasks — zsakib_ · 2026-10-16
- Muse Spark 816B gets llama.cpp support just 30 minutes after release — ngxson · 2026-10-16
- 24 AI Systems Recommend $314 Per Visit in Care vs Physicians' $97, a 3.2x Gap — dvir_a · 2026-10-16
- Are the fastest GPU kernels now written by AI? Kernels are hyper-verifiable, argues one dev — fleetwood___ · 2026-10-16
- Scale AI launches muse spark open: 816B-param MoE with 1M context, billed as top US open-weight model — jiayi_pirate · 2026-10-16
- NVIDIA HF listing for Qwen3.8-Flash-Next-FP4-Local leaks unannounced Qwen3.8 model — TheZachMueller · 2026-10-16