Sentry ran 10k+ AI evals in 90 days, over half in the past month alone
zeeg · x · 2026-11-20
Sentry engineer @rbro112 shared that the company's internal AI evaluation pipeline surpassed 10,000 eval runs over the last 90 days, with more than 50% of those happening in just the past month — a clear acceleration.
A dashboard screenshot shows the details: 10,004 runs, an 81.7% pass rate, and roughly 20.7B tokens consumed. Daily run volume has been steadily active since late August, peaking above 400 runs per day.
Co-founder zeeg amplified the stat. It's a concrete data point on how evals are becoming standard infrastructure for engineering teams adopting AI coding tools at scale.
More from coding & agent
- Free skill uses Blender and Fable 5.2 to un-slop vibe-coded websites from just a URL — nateliason · 2026-11-20
- Gemini API adds tool search preview: defer tool loading so agents skip sending full tool libraries — rseroter · 2026-11-20
- Builder on Reddit: Claude removed the friction between me and my tools — Live-Fault1571 · 2026-11-20
- Cognition: Gemini 4 Neon scores 46.2% on FrontierCode 2.0, near Opus 5.5 at 1/3 cost — scaling01 · 2026-11-20
- HKU Open-Sources PhoneGo, a CLI+GUI Hybrid Phone Agent Cutting Steps by 47% and Latency by 56% — aigclink · 2026-11-20
- Anthropic's AI code review: real-issue detection jumped from 16% to 54% — dotey · 2026-11-20