Grok 4.6 lands #2 on new DiligenceBench, statistically tied with Claude Opus 5
karinanguyen · x · 2026-08-19
DiligenceBench is a new equity-research agent eval jointly released by Paper Instruments and Thoughtful Lab: 150 open-ended diligence tasks across 10 sectors and 143 companies, with 5,428 rubric criteria measuring factual accuracy, analytical reasoning, and risk awareness. Tasks require external research (SEC filings) rather than recall, and the benchmark evaluates the full agent system—model, retrieval, tools, and instructions—across three harness tiers (H1 Loop / H2 Sandbox / H3 Finance).
Grok 4.6 enters the Finance harness leaderboard at #2 with 53.2, 1.1 points above Claude Opus 5 (52.1); the paired 95% CI of −1.0 to +3.2 means the two are statistically tied.
Behavioral differences stand out: Grok averaged 41 tool calls per task vs. 22 for Opus, and made 3,293 SEC filing searches vs. 486—broader search helped it find more evidence and follow task instructions better. Author Karina Nguyen argues the benchmark already captures some inference-time scaling (H1→H3), but more evals are needed to understand how performance scales with compute, search, and tool use.
Related event: Grok 4.6 Ties for Second on DiligenceBench at Low Cost(2 posts)→
More from Models
- Dev calls OpenAI Codex the best app ever, with the only models worth talking to — Zachly · 2026-08-19
- DeepSeek v4-flash: max thinking mode is both cheaper and faster than high — dosco · 2026-08-19
- User says still using model for code reviews, may switch to Grok — idanbeck · 2026-08-19
- Qwen 3.8 Comparison: Smaller Q4 Model Outreasons Larger Q5 — k-r-a-u-s-f-a-d-r · 2026-08-19
- GLM-5.3 ties Kimi K3 with score of 60, weights to be released — ArtificialAnlys · 2026-08-19
- Gemini Image Generation Silently Fails From Hetzner IPs — Network Origin Was the Culprit — dota2dinall · 2026-08-19