Thrive Holdings benchmarks AI on real tax and IT work across 75+ firms
BorisMPower · x · 2026-10-16
Thrive Holdings released its first benchmarks built on real work from 75+ accounting and IT services firms, measuring accuracy, time, and cost.
Individual tax prep (populating Form 1040 from client docs):
- Claude Opus 5.5 (high): 76.3% accuracy, $5.0/return, 6.1 min, 9.0M tokens
- Claude Fable 5.1 (max): 75.5% but $16.2 and 15.1 min
- GPT-6.1 Sol (medium): 74.2% at just $1.4 and 2.7 min — best cost/performance
Helpdesk ticket resolution:
- GPT-6 Astra (xhigh): 61.5% pass rate; Claude Opus 5.5 (max): 59.6%; GPT-6 Sol (max): 55.8%
Key takeaway: no single model wins everywhere; model choice is an operating decision trading off accuracy, time, and cost. The tests span 13 models across reasoning levels and will expand to more industries.
More from Models
- TabPFN sweeps Kaggle: now in 17% of public notebooks and 3 of September's top-10 solutions — ph_singer · 2026-10-16
- Gemini 4 Argon lands in Perplexity, claims new SoTA on WANDR benchmark — AravSrinivas · 2026-10-16
- DL Weekly #477: Haiku 5.5, building a coding agent from scratch, self-compacting agents paper — dl_weekly · 2026-10-16
- Dev says today's DLP tools are all regex-based, is rebuilding with LLM NER — HankYeomans · 2026-10-16
- Rumor: GLM 5.5 and Kimi K3.1 dropping soon, possibly slowed by anti-distillation blocks — zephyr_z9 · 2026-10-16
- Google Downgrading Prepaid Annual Gemini AI Plus Plans, Cutting Pro Access Nov 13 — Ray_ProUser · 2026-10-16